Kanban 1743: remote R script engine - #8019
Open
labkey-willm wants to merge 9 commits into
Open
Conversation
Runs R in a remote container instead of on the appserver. The working directory is tarred, staged to object storage, and handed to the runner as presigned URLs, so the runner holds no AWS credentials and no LabKey API key: each URL is scoped to one object and expires. DirectoryArchive lifts the tar create and extract logic out of DockerServiceImpl's private methods so it can be reused over a transport other than the Docker API. Extraction rejects entries resolving outside the destination, since the result tar comes back from a container that ran customer script code. Unit tests cover binary round-trip, input filtering and path traversal. RRemoteScriptEngine mirrors RDockerScriptEngine: the engine maps paths and delegates, the transport lives in an injected service registered by cloudServices. ExternalScriptEngineDefinition gains isRemoteRunner() to separate an HTTP runner from an Rserve host, since both are "remote". It rides the existing JSON configuration blob, so no schema change is needed. ScriptEngineManagerImpl throws rather than returning null when a definition targets a runner that is not configured, because returning null would silently fall back to running on the appserver, which is what this engine exists to avoid.
An empty prefix now yields unprefixed entries rather than ones beginning with a slash, so the archive can be extracted into any directory.
extract() dropped every entry whose name had no slash, silently discarding all remote results. Removes the parentName parameter as well, so create and extract cannot disagree about prefixing again.
The inherited epilog names the appserver's working directory, which does not exist in the container, so nothing was ever recorded.
It no longer affects tar entry names, only path mapping.
The runner selects an interpreter from the script extension, so nothing about the service contract is R-specific.
Follows the docker engine: the runner's endpoint and bucket are site-wide, so the engine form shows their status and links to the settings page rather than repeating them per engine.
They had a blank line between every line, unlike every other accessor in the file.
The script author decides how much the run writes, so an unbounded extract lets one report fill a disk shared by every application on the host. Caps total bytes and entries, rejects link entries, and rechecks the parent's real path because normalize() is lexical and cannot see a symlink already in the destination. Archiving no longer follows a symlinked directory: File.isDirectory() dereferences, so a link to any ancestor recursed until the stack or the disk gave out. A link to a file is still dereferenced and a broken one is skipped rather than failing the job. Remote runner failures now show the message alone. The chain can carry a presigned URL, which is a bearer credential for that object until it expires, and the text renders in the report pane.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale
Runs customer R scripts in a remote container instead of on the appserver. This is the engine and the archive handling; the transport lives in premiumModules.
Related Pull Requests
Changes
Design and measurements: https://claude.ai/code/artifact/3aab3803-490b-4e1e-8905-68923961be17