Fixed the private data reminder schedule and NCBI publication search failure handling

[vagisha]

Rationale

• The Private Data Reminder job stopped firing on production after a Tomcat restart.
• NCBI eutils can return a 5xx for requests, even when they are well under the rate limit. A failed search was indistinguishable from a dataset with no published paper.
• The job status reported COMPLETE even when there were errors in sending reminders to one or more datasets.

Related Pull Requests

• Search NCBI for papers associated with private datasets #606

Changes

• PanoramaPublicModule schedules the reminder job from startBackgroundThreads().
• NcbiPublicationSearchServiceImpl.getString retries eutils requests on 5xx, read timeouts and 429.
  • Up to three attempts, waiting 500ms before the second and 1000ms before the third.
  • HttpClient's own retry layer is disabled so the attempt count means what it says.
• searchForPublication throws NcbiSearchException only when no publication was found and at least one request failed, so an empty result still means no paper was found.
  • The reminder job records the failure against the experiment and still posts the reminder.
  • The two controller actions report an error rather than "no publications found".
• Added an optional NCBI API key to Private Data Reminder Settings, raising the eutils rate ceiling from 3 to 10 requests / second.
  • It is held in the encrypted property store, is never rendered back into the form
  • It can only be removed through the Remove the saved key checkbox, so editing the reminder settings cannot erase it.
  • The key is redacted from the retry warning and from NCBI's 4xx response body before either reaches a log.
  • NCBI echoes the submitted key back in its 400 body, and for the reminder job the log is the pipeline job log.
• PrivateDataReminderJob checks a configured NCBI API key before iterating over datasets.
  • The check is skipped when publication search is off or no key is saved.
  • If the key is rejected, the job is stopped with no reminders posted.
  • Only a 400 is a rejection. Any other status leaves the key unchecked. The job logs a warning and runs with the key anyway.
• Added a Validate button that checks a key against NCBI before it is saved.
  • The result is displayed beside the field with the rejection details behind a Details link.
  • A status that is not 400 is reported as a check that could not be completed.
• PrivateDataReminderJob.run reports a status computed from what the job recorded, rather than always reporting complete. An ERROR logged through the job's logger already sets the status to error, and the status set at the end was overwriting it.
  • ERROR means the job could not start, or a reminder that should have posted for a dataset was not.
  • A failed publication search is a WARN, since the reminder is still posted. It reports an error only when the search failed for more than half the datasets it ran for.
  • A missing support message thread is an error only when the submission has an announcement id. Older datasets, submitted before submission requests were posted to a message board, do not have an announcement id.

Tests

• NcbiPublicationSearchServiceImpl.TestCase
  • Retries 5xx, timeouts and 429, but not other 4xx.
  • A persistent 5xx is tried 3 times with 500ms then 1000ms backoff, then rethrown.
  • An interrupted thread stops after one attempt.
  • The key is stripped from the logged URL and from an echoed error body.
  • NCBI's reason reaches the log for a 4xx but not a 5xx.
  • A 400 from NCBI is a rejected key, while a 403, 404, 429, 5xx or timeout is a check that could not be completed.
• NcbiApiKeyTest - new Selenium test for the NCBI API key settings.
  • Fails with instructions if a key is already saved on the server, rather than overwriting one it cannot restore.
  • The saved key is never rendered into the form, and saving with the field blank keeps the stored key.
  • Validate button reports that a key was not accepted.
  • The Remove the saved key checkbox removes the key. Removes its own key in @After even if the test fails partway.
• PrivateDataReminderSettings.TestCase - the extension and reminder boundary tests now take the current date from the expiry the settings calculate, so they no longer fail on the last days of a month.

Co-Authored-By: Claude noreply@anthropic.com

[Copilot]

Pull request overview

Improves Panorama Public reminder scheduling and NCBI publication-search reliability.

Changes:

• Restores reminder scheduling during application startup.
• Adds NCBI retries and explicit search-failure handling.
• Adds encrypted API-key configuration, validation, and tests.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
PublicationSearchTest.java	Preserves reminder settings during testing.
PanoramaPublicBaseTest.java	Adds API-key settings test helpers.
NcbiApiKeyTest.java	Tests API-key management and validation.
privateDataRemindersSettingsForm.jsp	Adds API-key controls and feedback.
PrivateDataReminderJob.java	Handles key checks and search failures.
PanoramaPublicModule.java	Schedules reminders during startup.
PanoramaPublicController.java	Adds key validation and persistence.
NcbiSearchException.java	Distinguishes failed searches.
NcbiPublicationSearchServiceImpl.java	Adds retries, redaction, and key support.
NcbiPublicationSearchService.java	Extends the NCBI service contract.
NcbiApiKeyCheck.java	Models key-validation outcomes.
MockNcbiPublicationSearchService.java	Updates mocked request handling.
PrivateDataReminderSettings.java	Stores encrypted keys and stabilizes date tests.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[vagisha]

You are right. The UI search actions are request handlers and can overlap each other or the reminder job, so the comment claiming a retry never collides with a sibling request was wrong and has been corrected in bf13cb4

Not adding jitter. These are site-admin actions so collisions will be rare.