Skip to content

Use last rsd scrapers version v6.7.5 - #20

Merged
dmijatovic merged 2 commits into
mainfrom
scrapers
Nov 6, 2026
Merged

dmijatovic merged 2 commits into
mainfrom
scrapers

Conversation

@dmijatovic

@dmijatovic dmijatovic commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Use RSD scrapers service v6.7.5

Changes proposed in this pull request:

  • Copy RSD scrapers service v6.7.5 from RSD to KIN
  • Enable only "needed" scrapers services in crone for KIN. For scrapers we use latest v6.7.5 version.
  • Remove background-services as it is not used in KIN
  • Extend release script to create additional KIN images. We are switching from using RSD v3.4.1 to using KIN specific images with critical security patches.
  • Enable api/fe service to validate image version used

How to test:

  • make start to build and create test data

  • Validate scraper services. You will need to add 3 new .env variables MAX_REQUESTS_ROR=6, MAX_REQUESTS_OPENALEX=6 and OPENALEX_API_KEY=,

  • Create new test project

  • Wait few minutes and confirm all scraper services running without errors. Confirm that citations are scraped for the output you added in the previous step

  • Also check Admin/Logs, there should be no errors

  • Validate version using api: /api/fe should return v1.1.2 in this test

PR Checklist:

  • Increase version numbers in docker-compose.yml
  • Link to a GitHub issue
  • Update documentation
  • Tests

…ired scraper services

feat: enable api/fe version service to validate image version used
chore: extend release script
@dmijatovic
dmijatovic marked this pull request as ready for review November 5, 2026 13:29
@dmijatovic
dmijatovic requested a review from jmaassen November 5, 2026 13:29
@dmijatovic
dmijatovic force-pushed the scrapers branch 2 times, most recently from 6bfa9da to 77dec3e Compare November 6, 2026 12:40
fix: add healthchecks to services on starttime
chore: update deployment docker-compose.yml

@jmaassen jmaassen left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works as expected (provided the env variables are set correctly).

@dmijatovic
dmijatovic merged commit 7a30e59 into main Nov 6, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants