Custom dt environment prep by jr-rk · Pull Request #1303 · dataquest-dev/DSpace

jr-rk · 2026-04-13T20:58:36Z

Problem description

BE environment prep

Copilot

Pull request overview

Adds a set of local development / environment-prep scripts and introduces a new REST endpoint + scheduler to serve a cached, transformed Shibboleth DiscoFeed (IdP discovery feed) from the DSpace backend.

Changes:

Add numerous Windows/Linux helper scripts for building, restarting, indexing, and Docker (Matomo) management.
Add DiscoFeed backend implementation (controller + scheduler + download/transform service) plus sample response JSON.
Add an implementation guide describing the DiscoFeed setup and expected configuration.

Reviewed changes

Copilot reviewed 35 out of 35 changed files in this pull request and generated 32 comments.

Show a summary per file

File	Description
scripts/sourceversion.py	Prints build timestamp + git hash and outputs a build link from CLI args
scripts/run.delete.dspace.parent.bat	Wrapper to load env vars and delete/rebuild dspace-parent from local Maven repo
scripts/run.build.bat	Wrapper to load env vars and build/install DSpace into Tomcat/webapps
scripts/restart_debug/undebug.sh	Removes debug flag file and runs shutdown
scripts/restart_debug/redebug.sh	Creates debug flag file and runs shutdown
scripts/restart_debug/custom_run.sh	Starts Tomcat with/without JPDA based on a flag file
scripts/pre-commit/checkstyle.py	Pre-commit hook to run Maven Checkstyle
scripts/log4j2.solr.xml	Alternate Solr Log4j2 config (stdout WARN+, file logs info)
scripts/index-scripts/indexhandle.sh	Helper to run `dspace filter-media` for a handle and capture output/error
scripts/index-scripts/autoindexf.sh	Background wrapper around indexhandle + output management
scripts/fast-build/update-solr-configsets.bat	Batch helper to refresh Solr configsets from DSpace application
scripts/fast-build/tomcat/stop.bat	Stops Tomcat (fast-build tooling)
scripts/fast-build/tomcat/start.bat	Starts Tomcat with JPDA (fast-build tooling)
scripts/fast-build/oai-pmh-package-update.bat	Rebuilds and copies the dspace-oai jar into Tomcat
scripts/fast-build/dspace-api-package-update.bat	Rebuilds and copies the dspace-api jar into Tomcat and DSpace lib
scripts/fast-build/crosswalks-update.bat	Copies crosswalk configs and triggers OAI reindex
scripts/fast-build/config-update.bat	Copies full config directory into DSpace application
scripts/fast-build/cfg-update.bat	Copies selected cfg files into DSpace application
scripts/envs/__dspace.parent.basic.example.bat	Example env template for dspace-parent cleanup script
scripts/envs/__dspace.parent.basic.bat	Env file for dspace-parent cleanup script
scripts/envs/__basic.example.bat	Example env template for build/fast-build scripts
scripts/envs/__basic.bat	Env file for build/fast-build scripts
scripts/docker/matomo/startContainers.sh	Starts Matomo + DB stack via docker-compose
scripts/docker/matomo/startContainers.bat	Windows variant to start Matomo + DB stack
scripts/docker/matomo/matomo-w-db.yml	Docker Compose definition for Matomo + MariaDB + phpMyAdmin
scripts/docker/matomo/deleteContainers.sh	Stops/removes Matomo stack
scripts/docker/matomo/deleteContainers.bat	Windows variant to stop/remove Matomo stack
scripts/delete.dspace.parent.bat	Deletes dspace-parent directory and runs `mvn install`
scripts/build.dspace.bat	Runs Maven build + ant fresh_install + deploys webapps to Tomcat
scripts/__tests.yaml	Template YAML for generating test-run batch scripts
dspace-server-webapp/src/main/resources/org/dspace/app/rest/discofeedResponse.json	Sample DiscoFeed JSON response payload
dspace-server-webapp/src/main/java/org/dspace/app/rest/DiscoFeedsUpdateScheduler.java	Scheduled cache refresher for DiscoFeed content
dspace-server-webapp/src/main/java/org/dspace/app/rest/DiscoFeedsDownloadService.java	Downloads/transforms DiscoFeed JSON (with optional SSL bypass)
dspace-server-webapp/src/main/java/org/dspace/app/rest/DiscoFeedsController.java	Public REST endpoint serving cached DiscoFeed JSON
DISCOFEED-IMPLEMENTATION.md	Detailed guide for DiscoFeed endpoint architecture/configuration

Copilot · 2026-04-13T21:08:06Z

+echo going to index handle $1 with "-f"
+./dspace filter-media -f -i $1 > $1idx.out 2> $1idx.err


The shell redirections and argument uses are unquoted ($1, $1idx.out, etc.). If the handle contains spaces or shell metacharacters, this can break the command or write to unexpected paths. Quote the variable expansions when invoking dspace and when forming output filenames.

Suggested change

echo going to index handle $1 with "-f"

./dspace filter-media -f -i $1 > $1idx.out 2> $1idx.err

echo going to index handle "$1" with "-f"

./dspace filter-media -f -i "$1" > "${1}idx.out" 2> "${1}idx.err"

Copilot · 2026-04-13T21:08:06Z

@@ -0,0 +1,3 @@
+#!/bin/bash
+rm do_debug.txt


rm do_debug.txt will exit non-zero if the file doesn’t exist, which may break automated workflows that use this script. Use rm -f (or guard with test -f) to make it idempotent.

Copilot · 2026-04-13T21:08:06Z

@@ -0,0 +1,2 @@
+docker-compose -f matomo-w-db.yml down
+docker compose -f matomo-w-db.yml rm


docker-compose and docker compose are mixed across these scripts. Standardize on one to avoid failures on machines that only have one of them installed.

Suggested change

docker compose -f matomo-w-db.yml rm

docker-compose -f matomo-w-db.yml rm

Copilot · 2026-04-13T21:08:07Z

+@RestController
+@RequestMapping("/api/discojuice/feeds")
+public class DiscoFeedsController {
+
+    @Autowired
+    private DiscoFeedsUpdateScheduler discoFeedsUpdateScheduler;
+
+    /**
+     * Returns the cached IdP feed as a JSON array.
+     *
+     * @return HTTP 200 with the JSON feed, or HTTP 503 if the feed is not yet available.
+     */
+    @GetMapping(produces = MediaType.APPLICATION_JSON_VALUE)
+    @PreAuthorize("permitAll()")
+    public ResponseEntity<String> getDiscoFeeds() {


The PR title/description focuses on “environment prep”, but this change set also introduces a new public REST endpoint and scheduled background fetch with optional SSL validation bypass. Please update the PR description/scope so reviewers understand and can assess the functional/security impact.

Copilot · 2026-04-13T21:08:07Z

+rem set those to your local paths in .gitignored file /envs/__basic.bat.
+rem you HAVE TO CREATE /envs/__basic.bat with following variables in the same directory
+
+rem start of variables expected in /envs/__basic.bat
+set dspace_source=
+set tomcat=
+set dspace_application=
+rem end of variables expected in /envs/__basic.bat
+
+call envs\__basic.bat


The file scripts/envs/__basic.bat is committed, but scripts/run.build.bat comments imply the env file should be gitignored and user-specific. Please either remove the committed env file or adjust the script/documentation so the repository doesn’t include local machine settings.

Copilot · 2026-04-13T21:08:14Z

+    public String downloadAndTransformFeeds() {
+        disableSSL = configurationService.getBooleanProperty(
+                "disable.ssl.check.specific.requests", false);
+        String feedUrl = configurationService.getProperty("shibboleth.discofeed.url");


The config key disable.ssl.check.specific.requests is referenced here but does not appear to be defined/documented in the standard DSpace config files. Please either document it alongside the new DiscoFeed settings or switch to a DiscoFeed-specific key to avoid an undocumented global toggle.

Copilot · 2026-04-13T21:08:14Z

+
+rem copy created jar into tomcat/webapps/server
+xcopy /e /h /i /q /y %dspace_source%\dspace-api\target\dspace-api-7.6.1.jar %tomcat%\webapps\server\WEB-INF\lib\
+xcopy /e /h /i /q /y %dspace_source%\dspace-api\target\dspace-api-7.6.1.jar %dspace_application%\lib\


The jar filename is hardcoded to dspace-api-7.6.1.jar, which is unlikely to exist for this repo/version and will cause the copy step to fail. Use the Maven project version (or a wildcard) when copying the built jar.

Suggested change

rem copy created jar into tomcat/webapps/server

xcopy /e /h /i /q /y %dspace_source%\dspace-api\target\dspace-api-7.6.1.jar %tomcat%\webapps\server\WEB-INF\lib\

xcopy /e /h /i /q /y %dspace_source%\dspace-api\target\dspace-api-7.6.1.jar %dspace_application%\lib\

set "DSPACE_API_JAR_PATTERN=dspace-api-*.jar"

rem copy created jar into tomcat/webapps/server

xcopy /e /h /i /q /y %dspace_source%\dspace-api\target\%DSPACE_API_JAR_PATTERN% %tomcat%\webapps\server\WEB-INF\lib\

xcopy /e /h /i /q /y %dspace_source%\dspace-api\target\%DSPACE_API_JAR_PATTERN% %dspace_application%\lib\

Copilot · 2026-04-13T21:08:14Z

+# optional
+# debug: true - optional, use for debug
+# comment - optional
+# methondName - optional -use if want to run only one method


Typo in comment key name: methondName should be methodName (to match the actual YAML key used below).

Suggested change

# methondName - optional -use if want to run only one method

# methodName - optional -use if want to run only one method

Copilot · 2026-04-13T21:08:15Z

+
+    private static final Logger log = LogManager.getLogger(DiscoFeedsUpdateScheduler.class);
+
+    private String feedsContent;


feedsContent is written by the scheduled task thread and read by web request threads, but it’s not volatile and no synchronization is used. This can lead to stale reads under the Java memory model. Use volatile (or an AtomicReference) for the cached value.

Suggested change

private String feedsContent;

private volatile String feedsContent;

Copilot · 2026-04-13T21:08:15Z

+[
+  {
+    "entityID": "https://idp.example.org/idp/shibboleth",
+    "DisplayNames": [
+      {"value": "Example University", "lang": "en"}
+    ]
+  },
+  {
+    "entityID": "https://idp2.example.org/idp/shibboleth",
+    "DisplayNames": [
+      {"value": "Test University", "lang": "en"}
+    ]
+  }
+]


This JSON file looks like sample/test data, but it is added under src/main/resources and will be shipped with the production webapp. If it’s only for the TEST: classpath mode or for tests/docs, move it to src/test/resources (or otherwise document why it must be in the main artifact).

Suggested change

[

{

"entityID": "https://idp.example.org/idp/shibboleth",

"DisplayNames": [

{"value": "Example University", "lang": "en"}

]

},

{

"entityID": "https://idp2.example.org/idp/shibboleth",

"DisplayNames": [

{"value": "Test University", "lang": "en"}

]

}

]

[]

jr-rk added 3 commits April 13, 2026 17:40

add helpful scripts folder

34ebbfd

set __basic.bat env variables to be correct

06ee565

add discojuice config

825a394

jr-rk self-assigned this Apr 13, 2026

jr-rk requested a review from Copilot April 13, 2026 21:00

Copilot started reviewing on behalf of jr-rk April 13, 2026 21:01 View session

Copilot AI reviewed Apr 13, 2026

View reviewed changes

update local.cfg.EXAMPLE file to save Shibboleth configurations

e1c0ac8

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Custom dt environment prep#1303

Custom dt environment prep#1303
jr-rk wants to merge 4 commits intocustom_dtfrom
custom_dt_environment_prep

jr-rk commented Apr 13, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Apr 13, 2026

Uh oh!

Copilot AI Apr 13, 2026

Uh oh!

Copilot AI Apr 13, 2026

Uh oh!

Copilot AI Apr 13, 2026

Uh oh!

Copilot AI Apr 13, 2026

Uh oh!

Copilot AI Apr 13, 2026

Uh oh!

Copilot AI Apr 13, 2026

Uh oh!

Copilot AI Apr 13, 2026

Uh oh!

Copilot AI Apr 13, 2026

Uh oh!

Copilot AI Apr 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		echo going to index handle $1 with "-f"
		./dspace filter-media -f -i $1 > $1idx.out 2> $1idx.err

		@@ -0,0 +1,2 @@
		docker-compose -f matomo-w-db.yml down
		docker compose -f matomo-w-db.yml rm

	docker compose -f matomo-w-db.yml rm
	docker-compose -f matomo-w-db.yml rm

	# methondName - optional -use if want to run only one method
	# methodName - optional -use if want to run only one method


		private static final Logger log = LogManager.getLogger(DiscoFeedsUpdateScheduler.class);

		private String feedsContent;

	private String feedsContent;
	private volatile String feedsContent;

Conversation

jr-rk commented Apr 13, 2026

Problem description

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Apr 13, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants