Repository navigation
Add log cache log streaming support - #1338
Conversation
606817f to
08fb796
Compare
4e11f4c to
b0ce71f
Compare
9105a92 to
06acac7
Compare
Kehrlann
left a comment
There was a problem hiding this comment.
The API introduced in #1265 was specifically designed to transition seamlessly from Doppler to Logcache.
The idea was to:
- Deprecate the old
log(...)which used Doppler types and the Doppler API - Introduce a new API that is cache-system independent, put Doppler in the beginning but silently swap Logcache later on
We are fine doing this, because if users care about the underlying logger implementation, they should use DopplerClient or LogCacheClient directly ; "applications" is an implementation-agnostic wrapper.
#1348 was specifically built on top of this.
This PR should continue the work, and evolve logs to use LogCacheClient's new tailing API to stream logs.
We should not introduce new methods in Applications, nor expose LogCache specific APIs in Applications.
d4eced2 to
654697a
Compare
Signed-off-by: I539231 <pascal.zimmermann01@sap.com>
48561d2 to
47b07d9
Compare
I removed |
a1c1359 to
c0ec021
Compare
Summary
Adds live log streaming via Log Cache — the Java equivalent of
cf tail --follow.Previously the only Log Cache read path was
logsRecent, a single snapshot request. There was no way to continuously receive new log envelopes without polling manually. This change introduces alogsTailAPI across all three relevant modules.Motivation
Cloud Foundry dropped the legacy Loggregator Doppler streaming endpoint (
DopplerClient.stream()) in Loggregator ≥ 107.0 (CFD ≥ 24.3 / TAS ≥ 4.0). The CF CLI replaced it with a polling loop over the Log Cache/api/v1/readendpoint — that is whatcf tail --followdoes today. This PR brings the same capability to Java consumers.Changes
Changes in detail
cloudfoundry-client— newTailLogsRequestandLogCacheClient.logsTail()New
_TailLogsRequest.java(Immutables@Value.Immutable):sourceIdStringstartTimeLong(nullable)now − 5 s(ns)envelopeTypesList<EnvelopeType>(nullable)nameFilterString(nullable)pollIntervalDuration250 msNew method on
LogCacheClient:cloudfoundry-client-reactor— non-blocking polling implementationReactorLogCacheEndpoints.logsTail()mirrors the Go logcache.Walk() algorithm, fully non-blocking:AtomicLongcursor starts atstartTime(ornow − 5 s).Flux.deferbuilds a freshReadRequestfrom the current cursor on every repetition and callsGET /api/v1/read/{sourceId}?start_time=cursor.lastTimestamp + 1; each envelope is emitted individually downstream.repeatWheninserts aMono.delay(pollInterval)before the next poll. When envelopes arrive, the next poll starts immediately.Fluxis infinite; the caller cancels the subscription to stop.Transient network errors are logged and retried. HTTP errors (4xx/5xx) are propagated to the subscriber.
cloudfoundry-operations—Applications.logsTail()DefaultApplicationsdelegates toLogCacheClient:Tests
logsTailLogCacheLOG/OUTenvelope is forwarded correctlylogsTailLogCacheMultipleEnvelopesOUT → ERR → OUTare emitted in orderlogsTailLogCacheErrorRuntimeExceptionfrom the client propagates unchanged to the subscriberlogsTailLogCacheOutAndErrEnvelopesUsage
Or via the Operations API:
Relation to existing API
DopplerClient.stream()LogCacheClient.recentLogs()LogCacheClient.logsTail()✅ newChecklist
_TailLogsRequestImmutables value objectLogCacheClient.logsTail()interface methodReactorLogCacheEndpoints.logsTail()— non-blocking Reactor implementation_ReactorLogCacheClient.logsTail()— delegate overrideApplications.logsTail()— Operations API methodDefaultApplications.logsTail()— implementationDefaultApplicationsTest(120 passing)