diff --git a/DISCOFEED-IMPLEMENTATION.md b/DISCOFEED-IMPLEMENTATION.md new file mode 100644 index 000000000000..d500979e085d --- /dev/null +++ b/DISCOFEED-IMPLEMENTATION.md @@ -0,0 +1,366 @@ +# DiscoFeed / IdP Discovery Implementation Guide + +## Objective + +Implement a backend endpoint that serves a cached, transformed list of Identity Providers (IdPs) from the Shibboleth Service Provider's DiscoFeed. This powers a frontend IdP discovery widget for Shibboleth-based federated login. + +--- + +## Architecture Overview + +``` +Shibboleth SP DSpace Backend Frontend +┌──────────────┐ HTTP GET ┌─────────────────────┐ GET /api/ ┌──────────────┐ +│ /Shibboleth │ ◄──────────── │ DiscoFeedsDownload │ discojuice/ │ IdP Discovery│ +│ .sso/ │ (server-to- │ Service (fetches & │ feeds │ Widget │ +│ DiscoFeed │ server) │ transforms JSON) │ ◄──────────── │ │ +└──────────────┘ └─────────┬───────────┘ └──────┬───────┘ + │ │ + ▼ │ User picks IdP + ┌─────────────────────┐ │ + │ DiscoFeedsUpdate │ ▼ + │ Scheduler (cron │ Redirect to /Shibboleth.sso + │ cache refresh) │ /Login?entityID=...&target=... + └─────────┬───────────┘ + │ + ▼ + ┌─────────────────────┐ + │ DiscoFeedsController │ + │ GET /api/discojuice/ │ + │ feeds │ + └─────────────────────┘ +``` + +--- + +## What You Need to Create + +### 3 Java Files + +All three files go under `dspace-server-webapp/src/main/java/org/dspace/app/rest/` (pick a suitable sub-package, e.g., `repository/` or `discojuice/`). + +--- + +### 1. `DiscoFeedsController.java` + +**Purpose:** REST controller that serves the cached IdP feed JSON. + +**Requirements:** + +- `@RestController` with `@RequestMapping("/api/discojuice/feeds")` +- Single `GET` handler +- Must be publicly accessible — no authentication required (`@PreAuthorize("permitAll()")` or equivalent) +- Returns the cached JSON string from the scheduler (see below) +- Content-Type: `application/json` +- If cache is empty/null, return HTTP 503 or an empty JSON array `[]` + +**Pseudocode:** + +```java +@RestController +@RequestMapping("/api/discojuice/feeds") +public class DiscoFeedsController { + + @Autowired + private DiscoFeedsUpdateScheduler scheduler; + + @GetMapping(produces = MediaType.APPLICATION_JSON_VALUE) + @PreAuthorize("permitAll()") + public ResponseEntity getFeeds() { + String content = scheduler.getFeedsContent(); + if (StringUtils.isBlank(content)) { + return ResponseEntity.status(HttpStatus.SERVICE_UNAVAILABLE).body("[]"); + } + return ResponseEntity.ok(content); + } +} +``` + +--- + +### 2. `DiscoFeedsUpdateScheduler.java` + +**Purpose:** Background scheduled task that periodically fetches and caches the IdP feed. + +**Requirements:** + +- Spring `@Component` that implements `InitializingBean` (or use `@PostConstruct`) +- On startup: fetch the feed immediately (so the endpoint is populated before the first cron tick) +- On schedule: use `@Scheduled(cron = "${discojuice.refresh}")` to periodically refresh +- Must check `shibboleth.discofeed.allowed` config key — if `false`, skip fetching entirely +- Stores the fetched+transformed content in a `String` field (in-memory cache) +- Exposes a `getFeedsContent()` method for the controller + +**Config keys used:** + +| Key | Example Value | Purpose | +|-----|---------------|---------| +| `shibboleth.discofeed.allowed` | `true` | Feature toggle — if `false`, the feed is never fetched and the endpoint returns empty | +| `discojuice.refresh` | `0 */5 * * * *` | Cron expression for how often the feed is refreshed (every 5 minutes in this example) | + +**Pseudocode:** + +```java +@Component +public class DiscoFeedsUpdateScheduler implements InitializingBean { + + @Autowired + private ConfigurationService configurationService; + + @Autowired + private DiscoFeedsDownloadService downloadService; + + private String feedsContent; + + @Scheduled(cron = "${discojuice.refresh}") + public void refreshFeeds() { + boolean allowed = configurationService + .getBooleanProperty("shibboleth.discofeed.allowed", false); + if (!allowed) { + return; + } + feedsContent = downloadService.downloadAndTransformFeeds(); + } + + @Override + public void afterPropertiesSet() { + refreshFeeds(); // load on startup + } + + public String getFeedsContent() { + return feedsContent; + } +} +``` + +--- + +### 3. `DiscoFeedsDownloadService.java` + +**Purpose:** Fetches raw IdP metadata JSON from the Shibboleth SP's DiscoFeed endpoint, transforms it into a compact format, and returns it as a JSON string. + +**Requirements:** + +- Spring `@Service` +- Reads the DiscoFeed URL from config key `shibboleth.discofeed.url` +- Makes an HTTP GET request to that URL (server-to-server, typically `http://localhost/Shibboleth.sso/DiscoFeed`) +- Parses the response as a JSON array +- Transforms each IdP entry (shrink transform — see below) +- Deduplicates by `entityID` +- Returns the result as a JSON string (array of transformed IdP objects) + +**Config key used:** + +| Key | Example Value | Purpose | +|-----|---------------|---------| +| `shibboleth.discofeed.url` | `https://myserver.example.com/Shibboleth.sso/DiscoFeed` | URL of the Shibboleth SP's DiscoFeed handler | + +#### Raw Input Format (from Shibboleth SP) + +Each IdP entry in the raw DiscoFeed JSON array looks like: + +```json +{ + "entityID": "https://idp.example.org/idp/shibboleth", + "DisplayNames": [ + { "value": "Example University", "lang": "en" }, + { "value": "Ukázková Univerzita", "lang": "cs" } + ], + "Descriptions": [ + { "value": "IdP of Example University", "lang": "en" } + ], + "InformationURLs": [ ... ], + "Logos": [ + { "value": "https://...", "height": 16, "width": 16 }, + { "value": "https://...", "height": 80, "width": 80 } + ] +} +``` + +#### Shrink Transform + +For each IdP, produce a compact object: + +```json +{ + "entityID": "https://idp.example.org/idp/shibboleth", + "title": "Example University", + "country": "_all_" +} +``` + +**Transform rules:** + +1. Keep `entityID` as-is +2. Build `title` from `DisplayNames`: concatenate all `value` fields (e.g., `"Example University, Ukázková Univerzita"`) — or use the first one. This is what the frontend displays. +3. Set `country` to `"_all_"` (a static fallback — country-based filtering is optional and can be added later) +4. **Strip** all other fields: `Logos`, `InformationURLs`, `Descriptions`, `PrivacyStatementURLs` — these are not needed and add significant payload size +5. **Deduplicate** by `entityID` — if the same entityID appears more than once, keep only the first occurrence + +#### Expected Output Format + +```json +[ + { + "entityID": "https://idp.example.org/idp/shibboleth", + "title": "Example University", + "country": "_all_" + }, + { + "entityID": "https://idp2.example.org/idp/shibboleth", + "title": "Another University", + "country": "_all_" + } +] +``` + +**JSON parsing:** Use the `json-simple` library (`org.json.simple`), which is already a dependency of the `dspace-server-webapp` module. Alternatively, use Jackson (`com.fasterxml.jackson`) which is also available via Spring Boot. + +**HTTP client:** Use `java.net.HttpURLConnection`, Apache `HttpClient`, or Spring's `RestTemplate` / `WebClient` — whichever is idiomatic in the existing codebase. + +--- + +## Configuration Keys Summary + +Add these 3 keys to DSpace configuration (e.g., `local.cfg` or a dedicated module config file): + +```properties +# Enable/disable the DiscoFeed endpoint (must be true for the feed to work) +shibboleth.discofeed.allowed = true + +# URL of the Shibboleth SP's DiscoFeed handler (server-to-server) +shibboleth.discofeed.url = https://myserver.example.com/Shibboleth.sso/DiscoFeed + +# Cron expression for cache refresh (every 2 minutes in this example) +discojuice.refresh = 0 */2 * * * * +``` + +--- + +## Full Login Flow (Frontend → Backend → Shibboleth → Backend) + +This is the end-to-end flow the frontend widget initiates: + +1. **Frontend loads IdP list:** `GET /server/api/discojuice/feeds` → receives JSON array of IdPs +2. **User picks an IdP** from the widget (selects an `entityID`) +3. **Frontend redirects the browser** to: + ``` + /Shibboleth.sso/Login?entityID={URL-encoded-entityID}&target={URL-encoded-callback} + ``` + Where `target` is: + ``` + /server/api/authn/shibboleth?redirectUrl={URL-encoded-final-destination} + ``` +4. **Shibboleth SP** redirects the user to the selected IdP's login page +5. **User authenticates** at the IdP +6. **IdP posts SAML assertion** back to the Shibboleth SP +7. **Shibboleth SP** sets session headers and redirects to the `target` URL +8. **DSpace's `ShibbolethLoginFilter`** at `/api/authn/shibboleth` picks up the Shibboleth headers (`SHIB-*`, `eppn`, `mail`, etc.), creates/matches a DSpace EPerson, issues a JWT auth cookie +9. **DSpace redirects** to the `redirectUrl` parameter (the frontend page the user started from) + +### Redirect URL Construction (for the frontend) + +``` +https://{server}/Shibboleth.sso/Login + ?entityID={encodeURIComponent(selectedIdp.entityID)} + &target={encodeURIComponent( + "https://{server}/server/api/authn/shibboleth?redirectUrl=" + + encodeURIComponent(window.location.href) + )} +``` + +--- + +## Shibboleth SP Prerequisites + +The Shibboleth SP (`shibboleth2.xml`) must have: + +1. **DiscoFeed handler** enabled: + ```xml + + ``` + +2. **MetadataProvider(s)** configured — these determine which IdPs appear in the feed: + ```xml + + + ``` + Each `` element points to an IdP or federation metadata source. Only IdPs from configured providers appear in `/Shibboleth.sso/DiscoFeed`. + +3. **SSO element** that allows `entityID` override on the Login query string: + ```xml + + SAML2 + + ``` + The `entityID` attribute here sets the default IdP, but when the frontend passes `?entityID=...` on `/Shibboleth.sso/Login`, it overrides this default. + +--- + +## Existing DSpace Authentication Infrastructure + +These already exist in vanilla DSpace and do NOT need to be created: + +- **`ShibbolethLoginFilter`** — Spring Security filter at `/api/authn/shibboleth` that handles the Shibboleth callback (reads headers, creates EPerson, sets JWT cookie, redirects) +- **`ShibAuthentication`** — authentication plugin that processes Shibboleth attributes +- **`WebSecurityConfiguration`** — registers the Shibboleth login filter in the filter chain +- **`authentication-shibboleth.cfg`** — configuration for Shibboleth header mapping, lazy session, auto-registration + +The Shibboleth authentication module configuration (`authentication-shibboleth.cfg`) must be properly set: + +```properties +plugin.sequence.org.dspace.authenticate.AuthenticationMethod = org.dspace.authenticate.ShibAuthentication + +authentication-shibboleth.lazysession = true +authentication-shibboleth.lazysession.loginurl = /Shibboleth.sso/Login +authentication-shibboleth.netid-header = eppn +authentication-shibboleth.email-header = mail +authentication-shibboleth.autoregister = true +``` + +--- + +## Validation Checklist + +After implementation, verify: + +1. **Build succeeds:** `mvn clean install -DskipTests=true --no-transfer-progress -P-assembly` +2. **No compile errors** in `dspace-server-webapp` +3. **Controller is reachable:** `GET /server/api/discojuice/feeds` returns HTTP 200 with JSON array (or 503 if feed not yet loaded) +4. **Public access:** The endpoint does NOT require authentication +5. **Config toggles work:** Setting `shibboleth.discofeed.allowed = false` causes the endpoint to return `[]` or 503 +6. **Scheduler runs:** Confirm the cron expression fires and the cache is populated +7. **Startup load:** The feed is available immediately after application startup (no need to wait for first cron tick) +8. **JSON format:** Each entry has `entityID` (string), `title` (string), `country` (string) — no extra fields from the raw feed +9. **Deduplication:** No duplicate `entityID` values in the response +10. **Shibboleth login flow:** Browser redirect to `/Shibboleth.sso/Login?entityID=...&target=...` completes the SAML flow and returns to DSpace with auth cookie set +11. **Checkstyle passes:** `mvn checkstyle:check -f dspace-server-webapp/pom.xml --no-transfer-progress` +12. **No security issues:** The DiscoFeed URL should point to a trusted source (typically localhost or the same server); the endpoint doesn't expose sensitive data + +--- + +## File Placement Summary + +``` +dspace-server-webapp/src/main/java/org/dspace/app/rest/ + └── (pick sub-package, e.g., discojuice/) + ├── DiscoFeedsController.java + ├── DiscoFeedsUpdateScheduler.java + └── DiscoFeedsDownloadService.java + +dspace/config/local.cfg (or appropriate config location) + + shibboleth.discofeed.allowed = true + + shibboleth.discofeed.url = https://... + + discojuice.refresh = 0 */2 * * * * +``` + +--- + +## Important Notes + +- The endpoint path `/api/discojuice/feeds` is a convention from the reference implementation. You may adjust it, but the frontend must match. +- The `country` field is set to `"_all_"` as a static fallback. If country-based filtering is needed later, it can be derived from the IdP's metadata or GeoIP lookup. +- The `title` field is what the frontend displays to users. Include multiple language variants concatenated if available, or pick the primary language. +- The scheduler's cron expression `discojuice.refresh` uses Spring's 6-field cron format (seconds included): `second minute hour day month weekday`. +- If `@Scheduled` does not accept a property expression with a missing default gracefully, provide a sensible default: `@Scheduled(cron = "${discojuice.refresh:0 */5 * * * *}")`. diff --git a/dspace-server-webapp/src/main/java/org/dspace/app/rest/DiscoFeedsController.java b/dspace-server-webapp/src/main/java/org/dspace/app/rest/DiscoFeedsController.java new file mode 100644 index 000000000000..965f6cdf8179 --- /dev/null +++ b/dspace-server-webapp/src/main/java/org/dspace/app/rest/DiscoFeedsController.java @@ -0,0 +1,44 @@ +/** + * The contents of this file are subject to the license and copyright + * detailed in the LICENSE and NOTICE files at the root of the source + * tree and available online at + * + * http://www.dspace.org/license/ + */ +package org.dspace.app.rest; + +import org.apache.commons.lang3.StringUtils; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.http.HttpStatus; +import org.springframework.http.MediaType; +import org.springframework.http.ResponseEntity; +import org.springframework.security.access.prepost.PreAuthorize; +import org.springframework.web.bind.annotation.GetMapping; +import org.springframework.web.bind.annotation.RequestMapping; +import org.springframework.web.bind.annotation.RestController; + +/** + * REST controller that serves the cached IdP discovery feed JSON. + */ +@RestController +@RequestMapping("/api/discojuice/feeds") +public class DiscoFeedsController { + + @Autowired + private DiscoFeedsUpdateScheduler discoFeedsUpdateScheduler; + + /** + * Returns the cached IdP feed as a JSON array. + * + * @return HTTP 200 with the JSON feed, or HTTP 503 if the feed is not yet available. + */ + @GetMapping(produces = MediaType.APPLICATION_JSON_VALUE) + @PreAuthorize("permitAll()") + public ResponseEntity getDiscoFeeds() { + String feedsContent = discoFeedsUpdateScheduler.getFeedsContent(); + if (StringUtils.isBlank(feedsContent)) { + return ResponseEntity.status(HttpStatus.SERVICE_UNAVAILABLE).body("[]"); + } + return ResponseEntity.ok(feedsContent); + } +} \ No newline at end of file diff --git a/dspace-server-webapp/src/main/java/org/dspace/app/rest/DiscoFeedsDownloadService.java b/dspace-server-webapp/src/main/java/org/dspace/app/rest/DiscoFeedsDownloadService.java new file mode 100644 index 000000000000..af2ecbcf20f9 --- /dev/null +++ b/dspace-server-webapp/src/main/java/org/dspace/app/rest/DiscoFeedsDownloadService.java @@ -0,0 +1,179 @@ +/** + * The contents of this file are subject to the license and copyright + * detailed in the LICENSE and NOTICE files at the root of the source + * tree and available online at + * + * http://www.dspace.org/license/ + */ +package org.dspace.app.rest; + +import java.io.InputStream; +import java.net.URL; +import java.net.URLConnection; +import java.security.KeyManagementException; +import java.security.NoSuchAlgorithmException; +import java.security.SecureRandom; +import java.security.cert.X509Certificate; +import java.util.ArrayList; +import java.util.LinkedHashMap; +import java.util.List; +import java.util.Map; +import java.util.StringJoiner; +import javax.net.ssl.HttpsURLConnection; +import javax.net.ssl.SSLContext; +import javax.net.ssl.TrustManager; +import javax.net.ssl.X509TrustManager; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.node.ArrayNode; +import com.fasterxml.jackson.databind.node.ObjectNode; +import org.apache.commons.lang3.StringUtils; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.dspace.services.ConfigurationService; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.stereotype.Service; + +/** + * Service that fetches raw IdP metadata JSON from the Shibboleth SP DiscoFeed endpoint, + * transforms it into a compact format, and returns it as a JSON string. + */ +@Service +public class DiscoFeedsDownloadService { + + private static final Logger log = LogManager.getLogger(DiscoFeedsDownloadService.class); + + private static boolean disableSSL; + + private final ObjectMapper objectMapper = new ObjectMapper(); + + @Autowired + private ConfigurationService configurationService; + + /** + * Downloads the DiscoFeed JSON, applies the shrink transform, deduplicates by entityID, + * and returns the result as a JSON string. + * + * @return JSON string of transformed IdP entries, or null if download failed. + */ + public String downloadAndTransformFeeds() { + disableSSL = configurationService.getBooleanProperty( + "disable.ssl.check.specific.requests", false); + String feedUrl = configurationService.getProperty("shibboleth.discofeed.url"); + if (StringUtils.isBlank(feedUrl)) { + log.error("shibboleth.discofeed.url is not configured."); + return null; + } + + ArrayNode raw = downloadJSON(feedUrl); + if (raw == null || raw.isEmpty()) { + return null; + } + + // Shrink and deduplicate by entityID + Map seen = new LinkedHashMap<>(); + for (JsonNode node : raw) { + String entityID = node.path("entityID").asText(null); + if (entityID != null && !seen.containsKey(entityID)) { + seen.put(entityID, shrinkEntry(node)); + } + } + + List entries = new ArrayList<>(seen.values()); + try { + return objectMapper.writeValueAsString(entries); + } catch (Exception e) { + log.error("Failed to serialize DiscoFeed entries.", e); + return null; + } + } + + private ArrayNode downloadJSON(String url) { + try { + InputStream is; + if (url.startsWith("TEST:")) { + String classpathResource = url.substring("TEST:".length()); + is = getClass().getResourceAsStream(classpathResource); + if (is == null) { + log.error("Classpath resource not found: {}", classpathResource); + return null; + } + } else { + URLConnection conn = new URL(url).openConnection(); + if (conn instanceof HttpsURLConnection && disableSSL) { + disableCertificateValidation((HttpsURLConnection) conn); + } + conn.setConnectTimeout(5000); + conn.setReadTimeout(10000); + is = conn.getInputStream(); + } + try (is) { + JsonNode node = objectMapper.readTree(is); + if (node.isArray()) { + return (ArrayNode) node; + } + } + } catch (Exception e) { + log.error("Failed to download/parse DiscoFeed from {}", url, e); + } + return null; + } + + /** + * Disables SSL certificate validation on a specific HTTPS connection. + * This is for development / self-signed certificates only. + * Never applies globally — only to the supplied connection instance. + * + * @param connection the HTTPS connection to disable validation on. + */ + static void disableCertificateValidation(HttpsURLConnection connection) { + TrustManager[] trustAll = new TrustManager[] { + new X509TrustManager() { + @Override + public X509Certificate[] getAcceptedIssuers() { + return null; + } + @Override + public void checkClientTrusted(X509Certificate[] certs, String authType) { + // no-op: trust all for dev + } + @Override + public void checkServerTrusted(X509Certificate[] certs, String authType) { + // no-op: trust all for dev + } + } + }; + try { + SSLContext sc = SSLContext.getInstance("SSL"); + sc.init(null, trustAll, new SecureRandom()); + connection.setSSLSocketFactory(sc.getSocketFactory()); + connection.setHostnameVerifier((hostname, session) -> true); + } catch (NoSuchAlgorithmException | KeyManagementException e) { + throw new RuntimeException("Failed to disable SSL certificate validation", e); + } + } + + private ObjectNode shrinkEntry(JsonNode entity) { + ObjectNode compact = objectMapper.createObjectNode(); + compact.put("entityID", entity.path("entityID").asText("")); + compact.put("title", buildTitle(entity)); + compact.put("country", "_all_"); + return compact; + } + + private String buildTitle(JsonNode entity) { + JsonNode displayNames = entity.path("DisplayNames"); + if (displayNames.isMissingNode() || !displayNames.isArray()) { + return ""; + } + StringJoiner joiner = new StringJoiner(", "); + for (JsonNode nameNode : displayNames) { + String value = nameNode.path("value").asText(null); + if (value != null) { + joiner.add(value); + } + } + return joiner.toString(); + } +} \ No newline at end of file diff --git a/dspace-server-webapp/src/main/java/org/dspace/app/rest/DiscoFeedsUpdateScheduler.java b/dspace-server-webapp/src/main/java/org/dspace/app/rest/DiscoFeedsUpdateScheduler.java new file mode 100644 index 000000000000..72b968293256 --- /dev/null +++ b/dspace-server-webapp/src/main/java/org/dspace/app/rest/DiscoFeedsUpdateScheduler.java @@ -0,0 +1,66 @@ +/** + * The contents of this file are subject to the license and copyright + * detailed in the LICENSE and NOTICE files at the root of the source + * tree and available online at + * + * http://www.dspace.org/license/ + */ +package org.dspace.app.rest; + +import org.apache.commons.lang3.StringUtils; +import org.apache.logging.log4j.LogManager; +import org.apache.logging.log4j.Logger; +import org.dspace.services.ConfigurationService; +import org.springframework.beans.factory.InitializingBean; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.scheduling.annotation.Scheduled; +import org.springframework.stereotype.Component; + +/** + * Scheduled task that periodically fetches and caches the IdP discovery feed. + */ +@Component +public class DiscoFeedsUpdateScheduler implements InitializingBean { + + private static final Logger log = LogManager.getLogger(DiscoFeedsUpdateScheduler.class); + + private String feedsContent; + + @Autowired + private DiscoFeedsDownloadService discoFeedsDownloadService; + + @Autowired + private ConfigurationService configurationService; + + @Override + public void afterPropertiesSet() throws Exception { + refreshFeeds(); + } + + /** + * Fetch and cache the IdP discovery feed on a cron schedule. + */ + @Scheduled(cron = "${discojuice.refresh:-}") + public void refreshFeeds() { + boolean isAllowed = configurationService.getBooleanProperty("shibboleth.discofeed.allowed", false); + if (!isAllowed) { + return; + } + log.debug("Refreshing discovery feeds."); + String newContent = discoFeedsDownloadService.downloadAndTransformFeeds(); + if (StringUtils.isNotBlank(newContent)) { + feedsContent = newContent; + } else { + log.error("Failed to download discovery feeds."); + } + } + + /** + * Returns the cached feed content. + * + * @return JSON string of the IdP feed, or null if not yet loaded. + */ + public String getFeedsContent() { + return feedsContent; + } +} \ No newline at end of file diff --git a/dspace-server-webapp/src/main/resources/org/dspace/app/rest/discofeedResponse.json b/dspace-server-webapp/src/main/resources/org/dspace/app/rest/discofeedResponse.json new file mode 100644 index 000000000000..364c39c03a59 --- /dev/null +++ b/dspace-server-webapp/src/main/resources/org/dspace/app/rest/discofeedResponse.json @@ -0,0 +1,14 @@ +[ + { + "entityID": "https://idp.example.org/idp/shibboleth", + "DisplayNames": [ + {"value": "Example University", "lang": "en"} + ] + }, + { + "entityID": "https://idp2.example.org/idp/shibboleth", + "DisplayNames": [ + {"value": "Test University", "lang": "en"} + ] + } +] diff --git a/dspace/config/local.cfg.EXAMPLE b/dspace/config/local.cfg.EXAMPLE index 730aec8adc06..b1f46cda12bc 100644 --- a/dspace/config/local.cfg.EXAMPLE +++ b/dspace/config/local.cfg.EXAMPLE @@ -241,3 +241,26 @@ db.password = dspace # LDN INBOX SETTINGS # ######################## ldn.enabled = true + +#------------------------------------------------------------------# +#-----------------------DISCOFEED / SHIBBOLETH---------------------# +#------------------------------------------------------------------# + +# Master switch - set to true to enable the /api/discojuice/feeds endpoint +shibboleth.discofeed.allowed = true + +# Where to fetch the IdP list from. +# Option A: Use a test file (for local development without a Shibboleth SP): +# shibboleth.discofeed.url = TEST:/org/dspace/app/rest/discofeedResponse.json +# Option B: Use a real Shibboleth SP (for staging/production): +shibboleth.discofeed.url = https://lindat.mff.cuni.cz/Shibboleth.sso/DiscoFeed + +# How often to refresh the cached feed (Spring cron = seconds min hour day month weekday) +# This means: every 2 hours +discojuice.refresh = 0 0 */2 * * ? + +# List of entityIDs whose country should be rewritten (can be empty placeholders) +discojuice.rewriteCountries = https://idp.scc.kit.edu/idp/shibboleth + +# Disable SSL certificate check for the discofeed URL (useful for self-signed certs in dev) +disable.ssl.check.specific.requests = true \ No newline at end of file diff --git a/scripts/__tests.yaml b/scripts/__tests.yaml new file mode 100644 index 000000000000..3bb82c99db09 --- /dev/null +++ b/scripts/__tests.yaml @@ -0,0 +1,31 @@ +# copy this file and name it tests.yaml + +# If you want more test .bats generated, add another entry. +# Each entry generates its own .bat file + +# parameters description: +# mandatory +# source folder - part of dspace where tests should be run - required +# class name - required, without .java + +# optional +# debug: true - optional, use for debug +# comment - optional +# methondName - optional -use if want to run only one method + +# when ready, click regenerate.bat + +# WARNING - all .bats not specified in this file will be removed +# (when you no longer need ) + +tests: +# example entry #1 - with debug, whole class and with comment + - sourceFolder: "dspace-server-webapp" + className: "org.dspace.app.oai.OpenSearchControllerIT" + debug: true + comment: This is sample commentary + +# example entry #2 - testing one method + - sourceFolder: "dspace-server-webapp" + className: "org.dspace.app.oai.OAIpmhIT" + methodName: "listSetsWithMoreSetsThenMaxSetsPerPage" diff --git a/scripts/build.dspace.bat b/scripts/build.dspace.bat new file mode 100644 index 000000000000..6e66f4e8d74f --- /dev/null +++ b/scripts/build.dspace.bat @@ -0,0 +1,6 @@ +cd %dspace_source% +call mvn clean package +cd %dspace_installer% || echo 'failure' +call ant fresh_install +rd /s /q %server% +xcopy /e /h /i /q /y %dspace_webapps% %tomcat_webapps% diff --git a/scripts/delete.dspace.parent.bat b/scripts/delete.dspace.parent.bat new file mode 100644 index 000000000000..5c02bfbe6126 --- /dev/null +++ b/scripts/delete.dspace.parent.bat @@ -0,0 +1,3 @@ +IF EXIST %dspace_parent% rmdir %dspace_parent% /q /s +cd %dspace_source% || echo 'failure' +call mvn install \ No newline at end of file diff --git a/scripts/docker/matomo/deleteContainers.bat b/scripts/docker/matomo/deleteContainers.bat new file mode 100644 index 000000000000..88e34ec0c047 --- /dev/null +++ b/scripts/docker/matomo/deleteContainers.bat @@ -0,0 +1,2 @@ +docker-compose -f matomo-w-db.yml down +docker compose -f matomo-w-db.yml rm diff --git a/scripts/docker/matomo/deleteContainers.sh b/scripts/docker/matomo/deleteContainers.sh new file mode 100644 index 000000000000..88e34ec0c047 --- /dev/null +++ b/scripts/docker/matomo/deleteContainers.sh @@ -0,0 +1,2 @@ +docker-compose -f matomo-w-db.yml down +docker compose -f matomo-w-db.yml rm diff --git a/scripts/docker/matomo/matomo-w-db.yml b/scripts/docker/matomo/matomo-w-db.yml new file mode 100644 index 000000000000..a3332022b03e --- /dev/null +++ b/scripts/docker/matomo/matomo-w-db.yml @@ -0,0 +1,35 @@ +version: "3.5" + +services: + db: + image: mariadb + restart: always + ports: + - 127.0.0.1:3306:3306 + container_name: mdb + environment: + MARIADB_ROOT_PASSWORD: example + MARIADB_AUTO_UPGRADE: 1 + MARIADB_INITDB_SKIP_TZINFO: 1 + + gui: + image: phpmyadmin/phpmyadmin + ports: + - 8148:80 + container_name: phpAdmin + restart: always + links: + - "db:db" + + matomo: + image: matomo + container_name: matomo_statistics + restart: always + environment: + MATOMO_DATABASE_ADAPTER: mysql + MATOMO_DATABASE_HOST: db + MATOMO_DATABASE_USERNAME: root + MATOMO_DATABASE_PASSWORD: example + MATOMO_DATABASE_DBNAME: matomo_statistics + ports: + - 8135:80 diff --git a/scripts/docker/matomo/startContainers.bat b/scripts/docker/matomo/startContainers.bat new file mode 100644 index 000000000000..d144b8f69830 --- /dev/null +++ b/scripts/docker/matomo/startContainers.bat @@ -0,0 +1 @@ +docker-compose -f matomo-w-db.yml up diff --git a/scripts/docker/matomo/startContainers.sh b/scripts/docker/matomo/startContainers.sh new file mode 100644 index 000000000000..d144b8f69830 --- /dev/null +++ b/scripts/docker/matomo/startContainers.sh @@ -0,0 +1 @@ +docker-compose -f matomo-w-db.yml up diff --git a/scripts/envs/__basic.bat b/scripts/envs/__basic.bat new file mode 100644 index 000000000000..6d46cdbd6b43 --- /dev/null +++ b/scripts/envs/__basic.bat @@ -0,0 +1,5 @@ +set dspace_source=E:\workspace\DSpace-bc +set tomcat=E:\workspace\apache-tomcat-10.1.48 +set dspace_application=E:\dspace +set m2_source=%USERPROFILE%\.m2 +set dspace_solr=E:\workspace\solr diff --git a/scripts/envs/__basic.example.bat b/scripts/envs/__basic.example.bat new file mode 100644 index 000000000000..bc50cb505244 --- /dev/null +++ b/scripts/envs/__basic.example.bat @@ -0,0 +1,7 @@ +rem Set those paths to relevant places in your computer, copy this file and rename the copy to "__basic.bat" +set dspace_source=C:\workspace\DSpace\ +set tomcat=C:\apache-tomcat-9.0.64\ +set dspace_application=C:\dspace\ +set m2_source=%USERPROFILE%\.m2 +set dspace_solr=C:\workspace\solr +set dspace_source=C:\workspace\DSpace \ No newline at end of file diff --git a/scripts/envs/__dspace.parent.basic.bat b/scripts/envs/__dspace.parent.basic.bat new file mode 100644 index 000000000000..bb961632ddcc --- /dev/null +++ b/scripts/envs/__dspace.parent.basic.bat @@ -0,0 +1,3 @@ +rem Set those paths to relevant places in your computer, copy this file and rename the copy to "dspace.parent.basic.bat" +set m2_source= +set dspace_source= \ No newline at end of file diff --git a/scripts/envs/__dspace.parent.basic.example.bat b/scripts/envs/__dspace.parent.basic.example.bat new file mode 100644 index 000000000000..3ba2713726c3 --- /dev/null +++ b/scripts/envs/__dspace.parent.basic.example.bat @@ -0,0 +1,3 @@ +rem Set those paths to relevant places in your computer, copy this file and rename the copy to "dspace.parent.basic.bat" +set m2_source=C:\.m2 +set dspace_source=C:\workspace\DSpace \ No newline at end of file diff --git a/scripts/fast-build/cfg-update.bat b/scripts/fast-build/cfg-update.bat new file mode 100644 index 000000000000..8ce192650356 --- /dev/null +++ b/scripts/fast-build/cfg-update.bat @@ -0,0 +1,9 @@ +call ..\envs\__basic.bat + +call tomcat\stop.bat + +rem copy specific config files +xcopy /e /h /i /q /y %dspace_source%\dspace\config\clarin-dspace.cfg %dspace_application%\config\ +xcopy /e /h /i /q /y %dspace_source%\dspace\config\dspace.cfg %dspace_application%\config\ + +cd %dspace_source%\scripts\fast-build\ diff --git a/scripts/fast-build/config-update.bat b/scripts/fast-build/config-update.bat new file mode 100644 index 000000000000..b3bb9025e0dc --- /dev/null +++ b/scripts/fast-build/config-update.bat @@ -0,0 +1,10 @@ +call ..\envs\__basic.bat + +call tomcat\stop.bat + +rem copy all config files +xcopy /e /h /i /q /y %dspace_source%\dspace\config\ %dspace_application%\config\ + +cd %dspace_source%\scripts\fast-build\ + +REM call update-solr-configsets.bat diff --git a/scripts/fast-build/crosswalks-update.bat b/scripts/fast-build/crosswalks-update.bat new file mode 100644 index 000000000000..90e43624ec80 --- /dev/null +++ b/scripts/fast-build/crosswalks-update.bat @@ -0,0 +1,11 @@ +call ..\envs\__basic.bat + +call tomcat\stop.bat + +xcopy /e /h /i /q /y %dspace_source%\dspace\config\crosswalks\oai\metadataFormats\ %dspace_application%\config\crosswalks\oai\metadataFormats\ + +rem reindex oai-pmh indexes to show delete cache (sometimes the changes cannot be seen) +cd %dspace_application%\bin +call dspace oai import -c + +cd %dspace_source%\scripts\fast-build\ diff --git a/scripts/fast-build/dspace-api-package-update.bat b/scripts/fast-build/dspace-api-package-update.bat new file mode 100644 index 000000000000..3a01e579364c --- /dev/null +++ b/scripts/fast-build/dspace-api-package-update.bat @@ -0,0 +1,15 @@ +call ..\envs\__basic.bat + +rem stopping the tomcat +call tomcat\stop.bat + +rem rebuild the oai package (dspace-api) +cd %dspace_source%\dspace-api\ +call mvn clean package + +rem copy created jar into tomcat/webapps/server +xcopy /e /h /i /q /y %dspace_source%\dspace-api\target\dspace-api-7.6.1.jar %tomcat%\webapps\server\WEB-INF\lib\ +xcopy /e /h /i /q /y %dspace_source%\dspace-api\target\dspace-api-7.6.1.jar %dspace_application%\lib\ + +cd %dspace_source%\scripts\fast-build\ + diff --git a/scripts/fast-build/oai-pmh-package-update.bat b/scripts/fast-build/oai-pmh-package-update.bat new file mode 100644 index 000000000000..85363fd7d9b4 --- /dev/null +++ b/scripts/fast-build/oai-pmh-package-update.bat @@ -0,0 +1,17 @@ +call ..\envs\__basic.bat + +rem stopping the tomcat +call tomcat\stop.bat + +rem rebuild the oai package (dspace-oai) +cd %dspace_source%\dspace-oai\ +call mvn clean package + +rem copy created jar into tomcat/webapps/server +xcopy /e /h /i /q /y %dspace_source%\dspace-oai\target\dspace-oai-7.6.5.jar %tomcat%\webapps\server\WEB-INF\lib\ + +rem reindex oai-pmh indexes to show delete cache (sometimes the changes cannot be seen) +cd %dspace_application%\bin +call dspace oai import -c + +cd %dspace_source%\scripts\fast-build\ \ No newline at end of file diff --git a/scripts/fast-build/tomcat/start.bat b/scripts/fast-build/tomcat/start.bat new file mode 100644 index 000000000000..5c0e99c28ba6 --- /dev/null +++ b/scripts/fast-build/tomcat/start.bat @@ -0,0 +1,4 @@ +cd %tomcat%\bin\ +call catalina jpda run + +cd %dspace_source%\scripts\fast-build\tomcat\ diff --git a/scripts/fast-build/tomcat/stop.bat b/scripts/fast-build/tomcat/stop.bat new file mode 100644 index 000000000000..e24339d8ca88 --- /dev/null +++ b/scripts/fast-build/tomcat/stop.bat @@ -0,0 +1,8 @@ +call ..\..\envs\__basic.bat + +set tomcat_bin=%tomcat%\bin\ + +cd %tomcat_bin%\ +call catalina stop -force + +cd %dspace_source%\scripts\fast-build\tomcat\ diff --git a/scripts/fast-build/update-solr-configsets.bat b/scripts/fast-build/update-solr-configsets.bat new file mode 100644 index 000000000000..e5ca174ff29f --- /dev/null +++ b/scripts/fast-build/update-solr-configsets.bat @@ -0,0 +1,5 @@ +call ..\envs\__basic.bat + +rm -rf %dspace_solr%server\solr\configsets\authority %dspace_solr%server\solr\configsets\oai dspace_solr%server\solr\configsets\search dspace_solr%server\solr\configsets\statistics +xcopy /e /h /i /q /y %dspace_application%solr\ %dspace_solr%server\solr\configsets\ + diff --git a/scripts/index-scripts/autoindexf.sh b/scripts/index-scripts/autoindexf.sh new file mode 100644 index 000000000000..e9e07a7dfe0c --- /dev/null +++ b/scripts/index-scripts/autoindexf.sh @@ -0,0 +1,3 @@ +mkdir -p "${1%/*}" +./indexhandle $1 > $1manage.out 2> $1manage.err & +disown diff --git a/scripts/index-scripts/indexhandle.sh b/scripts/index-scripts/indexhandle.sh new file mode 100644 index 000000000000..36108788fdb0 --- /dev/null +++ b/scripts/index-scripts/indexhandle.sh @@ -0,0 +1,6 @@ +echo starting at +date +echo going to index handle $1 with "-f" +./dspace filter-media -f -i $1 > $1idx.out 2> $1idx.err +echo finished at +date diff --git a/scripts/log4j2.solr.xml b/scripts/log4j2.solr.xml new file mode 100644 index 000000000000..79e8b3f94a23 --- /dev/null +++ b/scripts/log4j2.solr.xml @@ -0,0 +1,86 @@ + + + + + + + + + + + %maxLen{%d{yyyy-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard} %X{replica} %X{core}] %c{1.} %m%notEmpty{ =>%ex{short}}}{10240}%n + + + + + + + + %maxLen{%d{yyyy-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard} %X{replica} %X{core}] %c{1.} %m%notEmpty{ =>%ex{short}}}{10240}%n + + + + + + + + + + + + + %maxLen{%d{yyyy-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard} %X{replica} %X{core}] %c{1.} %m%notEmpty{ =>%ex{short}}}{10240}%n + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/scripts/pre-commit/checkstyle.py b/scripts/pre-commit/checkstyle.py new file mode 100644 index 000000000000..3d7c8623da52 --- /dev/null +++ b/scripts/pre-commit/checkstyle.py @@ -0,0 +1,23 @@ +import sys +import logging +import subprocess + +_logger = logging.getLogger() +logging.basicConfig(format='%(message)s', level=logging.DEBUG) + + +if __name__ == '__main__': + files = [x for x in sys.argv[1:] if x.lower().endswith('java')] + _logger.info(f'Found [{len(files)}] files from [{len(sys.argv) - 1}] input files') + + cmd = "mvn checkstyle:check" + + try: + with subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, text=True) as process: + for line in process.stdout: + print(line, end='') + except Exception as e: + _logger.critical(f'Error: {repr(e)}, ret code: {e.returncode}') + + # for filename in files: + # pass diff --git a/scripts/restart_debug/custom_run.sh b/scripts/restart_debug/custom_run.sh new file mode 100644 index 000000000000..5fc45b1c0a15 --- /dev/null +++ b/scripts/restart_debug/custom_run.sh @@ -0,0 +1,9 @@ +#!/bin/bash +FILE=do_debug.txt +if [ -f "$FILE" ]; then + export JPDA_ADDRESS="*:8000" + ./catalina.sh jpda run +else + export JPDA_ADDRESS="localhost:8000" + ./catalina.sh run +fi diff --git a/scripts/restart_debug/redebug.sh b/scripts/restart_debug/redebug.sh new file mode 100644 index 000000000000..84ae4a08b08b --- /dev/null +++ b/scripts/restart_debug/redebug.sh @@ -0,0 +1,3 @@ +#!/bin/bash +touch do_debug.txt +./shutdown.sh diff --git a/scripts/restart_debug/undebug.sh b/scripts/restart_debug/undebug.sh new file mode 100644 index 000000000000..aab52a225961 --- /dev/null +++ b/scripts/restart_debug/undebug.sh @@ -0,0 +1,3 @@ +#!/bin/bash +rm do_debug.txt +./shutdown.sh diff --git a/scripts/run.build.bat b/scripts/run.build.bat new file mode 100644 index 000000000000..07a9ffc86a87 --- /dev/null +++ b/scripts/run.build.bat @@ -0,0 +1,17 @@ +rem set those to your local paths in .gitignored file /envs/__basic.bat. +rem you HAVE TO CREATE /envs/__basic.bat with following variables in the same directory + +rem start of variables expected in /envs/__basic.bat +set dspace_source= +set tomcat= +set dspace_application= +rem end of variables expected in /envs/__basic.bat + +call envs\__basic.bat + +set dspace_installer=%dspace_source%\dspace\target\dspace-installer\ +set tomcat_webapps=%tomcat%\webapps\ +set server=%tomcat_webapps%\server +set dspace_webapps=%dspace_application%\webapps\ +set tomcat_bin=%tomcat%\bin\ +call build.dspace.bat diff --git a/scripts/run.delete.dspace.parent.bat b/scripts/run.delete.dspace.parent.bat new file mode 100644 index 000000000000..99d5a622fbc8 --- /dev/null +++ b/scripts/run.delete.dspace.parent.bat @@ -0,0 +1,11 @@ +rem set those to your local paths in .gitignored file /envs/__dspace.parent.basic.bat. +rem you HAVE TO CREATE /envs/__dspace.parent.basic.bat with following variables in the same directory + +rem start of variables expected in /envs/__dspace.parent.basic.bat +set m2_source= +set dspace_source= + +call envs\__dspace.parent.basic.bat + +set dspace_parent=%m2_source%\repository\org\dspace\dspace-parent +call delete.dspace.parent.bat \ No newline at end of file diff --git a/scripts/sourceversion.py b/scripts/sourceversion.py new file mode 100644 index 000000000000..b838d70708c8 --- /dev/null +++ b/scripts/sourceversion.py @@ -0,0 +1,22 @@ +import subprocess +import sys +from datetime import datetime, timezone + +def get_time_in_timezone(zone: str = "Europe/Bratislava"): + try: + from zoneinfo import ZoneInfo + my_tz = ZoneInfo(zone) + except Exception as e: + my_tz = timezone.utc + return datetime.now(my_tz) + + +if __name__ == '__main__': + ts = get_time_in_timezone() + print(f"This info was generated on: {ts.strftime('%Y-%m-%d %H:%M:%S %Z%z')}") + + cmd = 'git log -1 --pretty=format:"Git hash: %H Date of commit: %ai"' + subprocess.check_call(cmd, shell=True) + + link = sys.argv[1] + sys.argv[2] + print(' Build run: ' + link + ' ')