Repository navigation
Conversation
1d389fa to
69dff25
Compare
69dff25 to
c9a1d95
Compare
|
I've applied @ossilator 's suggestions and rebased the PR on master to ease reviewing. |
38190ed to
5c2bd0d
Compare
Signed-off-by: Phil Krylov <phil@krylov.eu>
…pend scripts Signed-off-by: Phil Krylov <phil@krylov.eu>
Signed-off-by: Phil Krylov <phil@krylov.eu>
…L status Signed-off-by: Phil Krylov <phil@krylov.eu>
5c2bd0d to
b03acfb
Compare
|
Thanks for this PR! I merged it into my fork and tested it (on macOS with a fake 1. Every upload fails with "No supported send method". The new 2. Uploads are about 3 times slower than before. With both fixes, the upload is as fast as before, and your |
Proposed changes
I've hit #3128 and tried to fix it.
The old workflow during moving a file to a write-protected remote location looked like:
copy_file_file()callsmc_open()on the destination vpath.mc_open()callsshell_open()shell_open()generates a directory entry in the local VFS cache and returns a handle to it.copy_file_file()transfers the file content into the local temporary copy and setsdst_statustoDEST_FULL.copy_file_file()callsmc_close().mc_close()callsshell_file_store().shell_file_store()calls remote script,src/vfs/shell/helpers/send.sendanswers with### 001.shell_file_store()receives### 001and starts to send file data.sendfails to truncate the destination file with> "${FILENAME}", ignores the error, and starts the loop to consume exactly the specified number of bytes, attempting to write them to the write-protected destination path and ignoring any errors.sendanswers with### 200.mc_close()returns successfully.copy_file_file()tries to chmod and chown the file, fails, the user chooses to Ignore the errors.copy_file_file()returnsFILE_CONT, andmove_file_file()removes the source file, leaving no copy neither in source nor in destination.The simplest fix in the scenario with a write-protected destination is provided in the first patch, which adds an attempt to truncate the file or append 0 bytes to the file in
shell_open()before creating the fake directory entry in the local VFS cache, and returns an error on failure. It also has a positive side effect of failing early, before creating the local temporary copy etc.But the first patch alone does not solve the issue when the remote error occurs later during the copy, e.g. in an out of space situation. So the task was to make the
send(andappend) script detect and report the errors while also still working on the lowest common denominator of a shell environment. To do it, I had to implement some tests and some primitive test infrastructure for these scripts, which, as I've found out later, was already discussed in #3635. The test scripts are designed to run wherever the GitHub CI is running, they assume a fairly POSIX-compliant sh and are known to work indash,bash, and FreeBSD Almquist shell. It's the subject of the second patch.The third patch contains the updated
sendandappendscripts. They are written in an archaic dialect, so that Heirloom Bourne shell can run them. They handle the write errors, consume the rest of the data, and respond with### 500.But even when the scripts return
### 500andmc_close()returns an error,dst_statusis still equal toDEST_FULL, which is obviously a lie. The fourth patch catches this condition and changesdst_statustoDEST_SHORT_QUERY, asking the user to choose to keep or delete the partial file, and preventing the deletion of the source file inmove_file_file().* Needs: #5155.Well it's not a hard dependency, so I have rebased this PR on the master branch to ease reviewing.Checklist
git commit --amend -smake indent && make check)