Repository navigation
Python script to migrate non-SR projects - #1859
Conversation
rmunn
added
engineering
Unit Test Results362 tests 362 ✅ 8s ⏱️ Results for commit 313ee35. ♻️ This comment has been updated with latest results. |
Written by Claude Code (mostly Opus 5, some Opus 5.5), tested on local dev machine. Some projects, those with custom fields created in LF, will need a two-step process: run the script once, which will determine the XML to put into the FW project in order to define the custom fields. Then clone the Mercurial repo, add that XML as a new Mercurial commit, and push that commit to Lexbox. Then run the script again. The script is intended to run on the Language Forge `app` container, and that container cannot push Mercurial repos to LexBox, so the custom-fields commit needs to be a manual process. There are only six projects that look like they will need this, however, so it's not as onerous as it might sound. For all other projects, the script will run as-is without any manual steps. Once the script runs, there will be a `(projectCode)-migrate` project on the server with a copy of the project's data, and a new LexBox project with the same `(projectCode)-migrate` name that contains the project's data in FW format. That data can then be brought into FieldWorks Lite.
2344f04 to
b31b4f9
Compare
|
I am actually slightly disturbed that all checks have passed on this PR, because it shows there's a hole in our checks. I deliberately specified a version of the Probably not worth investing time to fix the checks on a server we're planning to sunset, but I did want to mention that the "mistake" in |
Lexbox initFwDataProject now returns HTTP 202 immediately and expects you to poll a different endpoint to verify that the project has been created and initialized. This updates the script to the new API shape.
Default timeout was 5 minutes to wait for project creation. That should be more than enough, but in theory, a server under heavy load might take longer, and I'd rather have a slower script that succeeds than a script that fails fast but would have succeeded had it waited longer.
This eliminates one unlikely-but-possible piece of human error when running the migration script with custom fields.
This script will copy the project to be migrated from the production server to your dev machine using the backup.mjs script, then run the Python migration script to push them to your local Lexbox instance. Needs Lexbox running locally (`task up`) and also a local copy of the web-languageforge Docker Compose (run `make` in the LF repo). Note that the Docker Compose file for LF is set up to run on port 8080 so that port 80 will belong to your local Lexbox ingress.
These are typical overengineered Claude-generated code, so they might be overkill. But they are designed to run in batches, keeping track of migration progress (which my hand-written script didn't do) so that the migration can be resumed easily after any failure. The custom-fields migration needs to proceed in two stages: first create the new repo, then the script pauses for you to copy a file into the repo, commit it, and push (this can't be done automatically since the LF container doesn't have Lexbox credentials), then step two is to run the Bash script again and it will finish the migration for those custom-fields projects. The non-custom-fields script is simpler: just run the script once and it should do everything.
The new custom-fields XML file needs to be added to the 750002.7000072 branch, not the default branch. Claude's generated instructions had omitted that step.
|
I've run extensive tests locally, and this is working. The necessary LfMerge bugfix PR has been merged and deployed to production. So it's time to merge this one as well. ... once it passes the tests again, that is. |
|
Playwright tests are failing because ubuntu-latest is suddenly ubuntu26.04 and our version of Playwright is old enough not to know about Ubuntu 26. (I think, I didn't check). I'm going to merge this despite the failing Playwright tests because it doesn't actually touch any LF code, it's just a script to be run by hand. |
Written by Claude Code (mostly Opus 5, some Opus 5.5), tested on local dev machine.
This is a Python script to migrate the projects identified in sillsdev/languageforge-lexbox#2414 as being worth migrating to LexBox and FW Lite. It needs sillsdev/LfMerge#361 and sillsdev/languageforge-lexbox#2439 to be merged first before the script can run.
Some projects, those with custom fields created in LF, will need a two-step process: run the script once, which will determine the XML to put into the FW project in order to define the custom fields. Then clone the Mercurial repo, add that XML as a new Mercurial commit, and push that commit to Lexbox. Then run the script again. The script is intended to run on the Language Forge
appcontainer, and that container cannot push Mercurial repos to LexBox, so the custom-fields commit needs to be a manual process. There are only six projects that look like they will need this, however, so it's not as onerous as it might sound.For all other projects, the script will run as-is without any manual steps. Once the script runs, there will be a
(projectCode)-migrateproject on the server with a copy of the project's data, and a new LexBox project with the same(projectCode)-migratename that contains the project's data in FW format. That data can then be brought into FieldWorks Lite.