Repository navigation
Fix duplicate telemetry recording for API calls - #10
Merged
Merged
Conversation
Each HTTP API call was recording TWO telemetry events because both UDP and TCP service layers called recordApiCall() independently when Promise.all ran them in parallel. Root cause: - handleApiCall in routes.ts called both service.getResponse() and tcpService.getResponse() via Promise.all - Each service method internally called telemetry.recordApiCall() - Result: false/0 event from the failing path, then true/Nms event from the successful path Fix: 1. Removed all telemetry recording from service layer (ASyncService and ASyncTCPService getResponse methods) 2. Added single telemetry recording at HTTP handler level in routes.ts 3. Record exactly ONE event per call: success with response time when device replies, failure on real timeout/error/not-connected 4. Improved response selection logic to prefer successful responses over error messages Added regression tests (telemetry-deduplication.test.ts): - Verify exactly one event per API call - Test success/failure scenarios - Test timeout and device-not-found cases - Test multiple concurrent calls Added historical data deduplication in analyse.ts: - Heuristic to de-duplicate false/0 followed by true within ~5s - Same device+method pairs from double-recording bug - Automatic cleanup when analyzing old telemetry data All tests pass, ASCII-only log output maintained. Co-authored-by: James Brown <JamesSmartCell@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Production telemetry showed each HTTP API call recording TWO
api_callevents:{apiSuccess: false, apiResponseTime: 0}{apiSuccess: true, apiResponseTime: 102}~165ms laterExample from production:
{apiMethod:'e2eTest', apiSuccess:false, apiResponseTime:0}at time t{apiMethod:'e2eTest', apiSuccess:true, apiResponseTime:102}at t+165msThis was for a single API call.
Root Cause
The
handleApiCallfunction inroutes.tscalls both UDP and TCP services in parallel:Each service method internally called
telemetry.recordApiCall(), resulting in:false/0eventtrue/102mseventSolution
1. Moved Telemetry Recording to HTTP Layer
Removed
recordApiCall()from:ASyncService.getResponse()(UDP layer)ASyncTCPService.getResponse()(TCP layer)Added single
recordApiCall()in:routes.tshandleApiCall()function (HTTP layer)Now records exactly ONE event per HTTP call with the final outcome.
2. Improved Response Selection Logic
Previously:
responseTCP ?? responseUDP(always preferred TCP even if it failed)Now: Smart selection that prefers successful responses:
Success defined as NOT:
'No device found','Timed out','API send error', or empty string.3. Added Regression Tests
New test file:
src/__tests__/telemetry-deduplication.test.tsTests verify:
4. Historical Data Deduplication
Added
deduplicateApiCallEvents()heuristic totools/telemetry/analyse.ts:false/0followed bytruefor same device+method within ~5 secondsfalse/0event, keeps only the successful oneTesting
Impact
Going forward:
Historical data:
analyse.tstool automatically de-duplicates old double-recorded events