[DO NOT REVIEW] Ignixa SDK Integration - CI E2E Test Run#5467
[DO NOT REVIEW] Ignixa SDK Integration - CI E2E Test Run#5467erikhoward wants to merge 23 commits intomainfrom
Conversation
- Added Microsoft.Health.Fhir.Ignixa project to Dockerfile for proper restore - Removed Linux_dotnet8 build jobs from pr-pipeline.yml and ci-pipeline.yml - Updated .vsts-PRInternalChecks-azureBuild-pipeline.yml to use net9.0 - Updated SDK version to 9.0.308 in global.json These changes are needed to complete the migration to .NET 9.0-only and ensure the Ignixa SDK integration works correctly in CI/CD pipelines. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
- FirelyCompiledFhirPath.cs: Add null-forgiving operator to ToString() - IgnixaCompiledFhirPath.cs: Convert if/else to ternary for result assignment - ImportResourceParser.cs: Fix null dereference by throwing when Parse returns null - IgnixaResourceValidator.cs: Use LINQ Where() for filtering issues by severity
- TokenSearchParamListRowGenerator: Handle empty/whitespace system strings by checking IsNullOrWhiteSpace instead of just null before calling GetSystemId - SqlServerFhirModel: Skip empty/whitespace values when loading System and QuantityCode caches to handle any legacy bad data in the database - IgnixaResourceValidator: Fall back to Firely validation for conformance resources (StructureDefinition, ValueSet, etc.) that have complex nested types not properly validated by Ignixa
Add unit tests for IgnixaFhirJsonInputFormatter, IgnixaFhirJsonOutputFormatter, and round-trip serialization fidelity (Ignixa <-> Firely). These are prerequisite coverage gates before further Ignixa migration work. - IgnixaFhirJsonInputFormatterTests: CanReadType, ResourceElement/Resource targets, Ignixa node preservation, error handling - IgnixaFhirJsonOutputFormatterTests: CanWriteType, Firely POCO output, RawResourceElement pass-through, pretty printing - IgnixaSerializationRoundTripTests: Firely->Ignixa->Firely fidelity for Patient/Observation/Coverage/Practitioner/Organization, Ignixa internal consistency, extensions/meta/contained edge cases - Investigation doc: current integration map, identified anti-patterns, phased improvement plan
- Add InternalsVisibleTo for Microsoft.Health.Fhir.R4.Benchmarks in both R4.Core and Core AssemblyInfo to access internal IgnixaFhirJsonInputFormatter and ResourceElement constructor - Fix namespace shadowing: use global::Ignixa.Serialization.SourceNodes.ResourceJsonNode in SerializationBenchmarks - Replace inaccessible FhirJsonInputFormatter with direct FhirJsonParser for Firely baseline in InputFormatterBenchmarks - Disable code analysis and test platform entry point generation in benchmark csproj
… report Phase 2 — Critical test coverage: - Add IgnixaFhirPathProviderTests (12 tests): compile, cache, evaluate, scalar, predicate - Add InputFormatter tests for ResourceJsonNode and IgnixaResourceElement targets - Add OutputFormatter tests for ResourceJsonNode and IgnixaResourceElement CanWrite + Write - Register IgnixaFhirPathProviderTests in Shared.Core.UnitTests.projitems Phase 3 — Eliminate high-impact fallbacks: - InputFormatter: replace Ignixa-serialize/Firely-parse with ITypedElement.ToPoco() - OutputFormatter: write Firely JSON directly instead of triple-hop - RawResourceFactory: use Firely JSON directly, skip Ignixa re-serialization Infrastructure: - global.json: update SDK to 9.0.312 with rollForward:latestPatch - Add ignixa-test-readiness-report.md with full test analysis
# Conflicts: # global.json
| // Try conversion for common types | ||
| if (value != null && typeof(T) == typeof(string)) | ||
| { | ||
| return (T)(object)value.ToString()!; |
Check warning
Code scanning / CodeQL
Useless upcast Warning
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 days ago
To fix the problem, remove the redundant upcast from string to object in the Scalar<T> method. The goal is to keep the existing semantics—only converting to a string when typeof(T) == typeof(string)—while eliminating the unnecessary (object) cast.
Concretely, in src/Microsoft.Health.Fhir.Core/Features/Search/FhirPath/FirelyCompiledFhirPath.cs, within the Scalar<T> method, locate the block:
if (value != null && typeof(T) == typeof(string))
{
return (T)(object)value.ToString()!;
}Change only the return statement so that it casts the string result directly to T without the intermediate cast to object:
if (value != null && typeof(T) == typeof(string))
{
return (T)value.ToString()!;
}No additional methods, imports, or definitions are needed. This removes the useless upcast while preserving behavior, since in this branch T is known (at runtime) to be string.
| @@ -61,7 +61,7 @@ | ||
| // Try conversion for common types | ||
| if (value != null && typeof(T) == typeof(string)) | ||
| { | ||
| return (T)(object)value.ToString()!; | ||
| return (T)value.ToString()!; | ||
| } | ||
|
|
||
| return default; |
| public async Task ParseAsync_FromStream_ReturnsResourceJsonNode() | ||
| { | ||
| // Arrange | ||
| var stream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(SamplePatientJson)); |
Check warning
Code scanning / CodeQL
Missing Dispose call on local IDisposable Warning
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 days ago
In general, the correct fix is to ensure that any IDisposable instance created in a method is disposed when it is no longer needed. The preferred pattern is to wrap its usage in a using (or await using for IAsyncDisposable) statement so that disposal happens automatically even if an exception is thrown.
Here, the single MemoryStream created in ParseAsync_FromStream_ReturnsResourceJsonNode should be disposed once _serializer.ParseAsync(stream) has completed. The best fix that does not change existing functionality is to wrap the MemoryStream creation and subsequent use in a using block. Because the method is async, we need the C# 8 using declaration or a standard using block; both correctly dispose the stream after the awaited call. To keep the change explicit and compatible, we can introduce a using block:
- Replace the standalone declaration
var stream = new MemoryStream(...); - With a
usingblock:using var stream = new MemoryStream(...);(or a block-formusing), then leave the rest of the method unchanged.
No new methods or imports are required; MemoryStream and System.Text.Encoding are already in use via fully-qualified names.
| @@ -142,7 +142,7 @@ | ||
| public async Task ParseAsync_FromStream_ReturnsResourceJsonNode() | ||
| { | ||
| // Arrange | ||
| var stream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(SamplePatientJson)); | ||
| using var stream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(SamplePatientJson)); | ||
|
|
||
| // Act | ||
| var resource = await _serializer.ParseAsync(stream); |
| // Try conversion for common types | ||
| if (value != null && typeof(T) == typeof(string)) | ||
| { | ||
| return (T)(object)value.ToString()!; |
Check warning
Code scanning / CodeQL
Useless upcast Warning
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 days ago
In general, to fix a "useless upcast" you remove the explicit cast to a base type when an implicit conversion already exists and the cast doesn’t affect overload resolution or generics. Here, value.ToString() returns string, and when typeof(T) == typeof(string) the cast (T)(object)... is effectively just a roundabout way of casting from string to T. You can safely remove the (object) and cast directly from string to T.
Concretely, in src/Microsoft.Health.Fhir.Ignixa/FhirPath/IgnixaCompiledFhirPath.cs, within the Scalar<T> method, change line 92 from:
return (T)(object)value.ToString()!;to:
return (T)value.ToString()!;No additional imports, methods, or definitions are needed; this is a purely local expression change that does not alter functionality.
| @@ -89,7 +89,7 @@ | ||
| // Try conversion for common types | ||
| if (value != null && typeof(T) == typeof(string)) | ||
| { | ||
| return (T)(object)value.ToString()!; | ||
| return (T)value.ToString()!; | ||
| } | ||
|
|
||
| return default; |
| foreach (var expr in expressions) | ||
| { | ||
| var compiled = _provider.Compile(expr); | ||
| var results = compiled.Evaluate(typedElement).ToList(); | ||
| Assert.NotEmpty(results); | ||
| } |
Check notice
Code scanning / CodeQL
Missed opportunity to use Select Note
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 days ago
In general, to fix this pattern you precompute the transformed sequence with LINQ Select and iterate over the transformed values directly, instead of manually transforming each item inside the loop. Here, the transformation is _provider.Compile(expr) from a string expression to a compiled FhirPath expression. We can either (a) perform Select inside the foreach header, or (b) extract a separate compiledExpressions sequence before the loop; both preserve functionality.
The minimal, clear change in this file is to update the foreach on lines 232–237 so that it iterates over compiled expressions rather than raw strings. We do this by replacing:
foreach (var expr in expressions)
{
var compiled = _provider.Compile(expr);
var results = compiled.Evaluate(typedElement).ToList();
Assert.NotEmpty(results);
}with:
foreach (var compiled in expressions.Select(expr => _provider.Compile(expr)))
{
var results = compiled.Evaluate(typedElement).ToList();
Assert.NotEmpty(results);
}System.Linq is already imported (line 6), so no new imports are necessary. The test behavior is unchanged: for each expression string, we still compile it and assert that evaluation results are non-empty; we just perform the compilation via Select rather than inside the loop body.
| @@ -229,9 +229,8 @@ | ||
| }; | ||
|
|
||
| // Act & Assert | ||
| foreach (var expr in expressions) | ||
| foreach (var compiled in expressions.Select(expr => _provider.Compile(expr))) | ||
| { | ||
| var compiled = _provider.Compile(expr); | ||
| var results = compiled.Evaluate(typedElement).ToList(); | ||
| Assert.NotEmpty(results); | ||
| } |
| foreach (var field in referenceMetadata) | ||
| { | ||
| if (string.IsNullOrWhiteSpace(reference.Reference)) | ||
| // Strip [x] suffix for choice types - FhirPath handles polymorphic fields | ||
| var elementPath = field.ElementPath.EndsWith("[x]", StringComparison.Ordinal) | ||
| ? field.ElementPath.Substring(0, field.ElementPath.Length - 3) | ||
| : field.ElementPath; | ||
|
|
||
| // Use FhirPath to check if any reference contains '?' (conditional reference) | ||
| // FhirPath naturally handles collections and nested paths | ||
| var fhirPath = $"{elementPath}.reference.contains('?')"; | ||
| if (ignixaElement.Predicate(fhirPath)) | ||
| { | ||
| continue; | ||
| throw new NotSupportedException($"Conditional reference is not supported for $import in {ImportMode.InitialLoad}."); | ||
| } | ||
| } |
Check notice
Code scanning / CodeQL
Missed opportunity to use Select Note
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 days ago
In general, to address this pattern, transform the loop so that it iterates directly over the mapped values instead of mapping inside the loop body. Here, referenceMetadata is only used to compute elementPath from field.ElementPath with an optional [x] suffix removal. We can map referenceMetadata to a sequence of elementPath strings using LINQ’s Select, and then iterate over that sequence.
Concretely in ImportResourceParser.CheckConditionalReferenceInResource, change the foreach (var field in referenceMetadata) loop to iterate over referenceMetadata.Select(...), where the selector produces the same elementPath string currently computed inside the loop. The body of the loop will then operate on elementPath directly, with no need for the field variable. This preserves all existing logic while making the mapping explicit. To use Select, we must ensure System.Linq is imported at the top of the file. No other methods, types, or behavior need to change.
| @@ -6,6 +6,7 @@ | ||
| using System; | ||
| using System.Text.Json.Nodes; | ||
| using System.Text.RegularExpressions; | ||
| using System.Linq; | ||
| using EnsureThat; | ||
| using Ignixa.Serialization.SourceNodes; | ||
| using Microsoft.Health.Core.Extensions; | ||
| @@ -85,13 +86,11 @@ | ||
|
|
||
| // Use IReferenceMetadataProvider to identify reference fields for this resource type | ||
| var referenceMetadata = _schemaContext.ReferenceMetadataProvider.GetMetadata(resource.ResourceType); | ||
| foreach (var field in referenceMetadata) | ||
| foreach (var elementPath in referenceMetadata.Select(field => | ||
| field.ElementPath.EndsWith("[x]", StringComparison.Ordinal) | ||
| ? field.ElementPath.Substring(0, field.ElementPath.Length - 3) | ||
| : field.ElementPath)) | ||
| { | ||
| // Strip [x] suffix for choice types - FhirPath handles polymorphic fields | ||
| var elementPath = field.ElementPath.EndsWith("[x]", StringComparison.Ordinal) | ||
| ? field.ElementPath.Substring(0, field.ElementPath.Length - 3) | ||
| : field.ElementPath; | ||
|
|
||
| // Use FhirPath to check if any reference contains '?' (conditional reference) | ||
| // FhirPath naturally handles collections and nested paths | ||
| var fhirPath = $"{elementPath}.reference.contains('?')"; |
| if (pretty) | ||
| { | ||
| json = await new FhirJsonSerializer(new SerializerSettings { Pretty = true }).SerializeToStringAsync(resource).ConfigureAwait(false); | ||
| } | ||
| else | ||
| { | ||
| json = await _firelySerializer.SerializeToStringAsync(resource).ConfigureAwait(false); | ||
| } |
Check notice
Code scanning / CodeQL
Missed ternary opportunity Note
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 days ago
In general, to fix this kind of issue, replace an if/else where both branches only assign to the same variable or return different values with a single return or assignment statement using the ternary (? :) operator. This shortens the code and makes it clear that the sole purpose of the condition is to select between two values.
Here, in WriteFirelyResourceAsync (around lines 204–218), we can eliminate the string json; declaration and the subsequent if (pretty)/else that assign to json. Instead, declare and initialize json in a single statement using the ternary operator, with each branch awaiting the appropriate async serialization call. The rest of the method (await response.WriteAsync(json, encoding) and FlushAsync) remains unchanged, so functionality is preserved.
Concretely, in src/Microsoft.Health.Fhir.Shared.Core/Ignixa/IgnixaFhirJsonOutputFormatter.cs, replace:
- The standalone declaration
string json; - The
if (pretty) { ... } else { ... }that assigns tojson
with a single line:
string json = pretty
? await new FhirJsonSerializer(new SerializerSettings { Pretty = true }).SerializeToStringAsync(resource).ConfigureAwait(false)
: await _firelySerializer.SerializeToStringAsync(resource).ConfigureAwait(false);No new imports, methods, or definitions are needed.
| @@ -203,15 +203,9 @@ | ||
| /// </remarks> | ||
| private async Task WriteFirelyResourceAsync(Resource resource, HttpResponse response, bool pretty, Encoding encoding) | ||
| { | ||
| string json; | ||
| if (pretty) | ||
| { | ||
| json = await new FhirJsonSerializer(new SerializerSettings { Pretty = true }).SerializeToStringAsync(resource).ConfigureAwait(false); | ||
| } | ||
| else | ||
| { | ||
| json = await _firelySerializer.SerializeToStringAsync(resource).ConfigureAwait(false); | ||
| } | ||
| string json = pretty | ||
| ? await new FhirJsonSerializer(new SerializerSettings { Pretty = true }).SerializeToStringAsync(resource).ConfigureAwait(false) | ||
| : await _firelySerializer.SerializeToStringAsync(resource).ConfigureAwait(false); | ||
|
|
||
| await response.WriteAsync(json, encoding).ConfigureAwait(false); | ||
| await response.Body.FlushAsync().ConfigureAwait(false); |
| foreach (var expr in Expressions) | ||
| { | ||
| var compiled = _firelyProvider.Compile(expr); | ||
| total += compiled.Evaluate(_firelyElement).Count(); | ||
| } |
Check notice
Code scanning / CodeQL
Missed opportunity to use Select Note test
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 days ago
In general, to fix this type of issue, replace a foreach loop that immediately and solely maps its iteration variable to another value with a foreach over the sequence produced by Select. This makes the transformation explicit and separates the mapping concern from subsequent processing.
Here, we can change each loop so that instead of:
foreach (var expr in Expressions)
{
var compiled = _firelyProvider.Compile(expr);
total += compiled.Evaluate(_firelyElement).Count();
}we iterate directly over the compiled expressions:
foreach (var compiled in Expressions.Select(expr => _firelyProvider.Compile(expr)))
{
total += compiled.Evaluate(_firelyElement).Count();
}Do the analogous change in Ignixa_EvaluateAll. This keeps behavior identical: Expressions is enumerated once, compilation happens once per expression in the same order, and total accumulates the counts as before. No new methods or imports are required; System.Linq is already imported at the top of FhirPathBenchmarks.cs.
Concretely:
- In
test/Microsoft.Health.Fhir.R4.Benchmarks/FhirPathBenchmarks.cs, replace the loop inFirely_EvaluateAll(lines 77–81) with aforeachoverExpressions.Select(...)using_firelyProvider. - Replace the loop in
Ignixa_EvaluateAll(lines 90–93) with aforeachoverExpressions.Select(...)using_ignixaProvider.
| @@ -74,9 +74,8 @@ | ||
| public int Firely_EvaluateAll() | ||
| { | ||
| int total = 0; | ||
| foreach (var expr in Expressions) | ||
| foreach (var compiled in Expressions.Select(expr => _firelyProvider.Compile(expr))) | ||
| { | ||
| var compiled = _firelyProvider.Compile(expr); | ||
| total += compiled.Evaluate(_firelyElement).Count(); | ||
| } | ||
|
|
||
| @@ -87,9 +85,8 @@ | ||
| public int Ignixa_EvaluateAll() | ||
| { | ||
| int total = 0; | ||
| foreach (var expr in Expressions) | ||
| foreach (var compiled in Expressions.Select(expr => _ignixaProvider.Compile(expr))) | ||
| { | ||
| var compiled = _ignixaProvider.Compile(expr); | ||
| total += compiled.Evaluate(_ignixaElement).Count(); | ||
| } | ||
|
|
| foreach (var expr in Expressions) | ||
| { | ||
| var compiled = _ignixaProvider.Compile(expr); | ||
| total += compiled.Evaluate(_ignixaElement).Count(); | ||
| } |
Check notice
Code scanning / CodeQL
Missed opportunity to use Select Note test
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 9 days ago
To fix this, we should express the transformation from expr to compiled using LINQ’s Select, then iterate over the resulting sequence of compiled expressions. This keeps behavior identical while making the intent—“iterate over compiled expressions for all entries in Expressions”—explicit.
Concretely, in Ignixa_EvaluateAll (lines 87–97), replace the foreach (var expr in Expressions) loop with a foreach over Expressions.Select(expr => _ignixaProvider.Compile(expr)). Inside the loop, we can then use the iteration variable (e.g., compiled) directly and remove the local compiled declaration. No new imports are needed because System.Linq is already imported at line 6. The rest of the method and file remains unchanged.
| @@ -87,9 +87,8 @@ | ||
| public int Ignixa_EvaluateAll() | ||
| { | ||
| int total = 0; | ||
| foreach (var expr in Expressions) | ||
| foreach (var compiled in Expressions.Select(expr => _ignixaProvider.Compile(expr))) | ||
| { | ||
| var compiled = _ignixaProvider.Compile(expr); | ||
| total += compiled.Evaluate(_ignixaElement).Count(); | ||
| } | ||
|
|
- 34 CI checks pass, 15 fail, 9 pending/not completed - SQL E2E BulkUpdate passes all 4 FHIR versions (validates full HTTP pipeline) - All Reindex E2E and Cosmos E2E fail due to RC-1 (mocked IFhirPathProvider) - R4/R4B Cosmos Integration fail due to RC-1 + RC-2 combined - Added P0/P1 fix plan with estimated 30min effort to unblock all failures
…tion gap Failure classifications: Fail-Ignixa-Production-Gap, Fail-Pre-existing, Fail-Test-Setup-Bug, Skip-Infrastructure Production gap found: IgnixaSchemaContext.cs uses R4CoreSchemaProvider for STU3/R4B STU3CoreSchemaProvider and R4BCoreSchemaProvider exist in Ignixa.Specification v0.0.163 FhirStorageTests(SqlServer) verified passing with Ignixa RawResourceFactory path
Ignixa Test Readiness — Failure Summary by Category
Validated (passing)
P0 Fix Required (production code)IgnixaSchemaContext.cs: use STU3CoreSchemaProvider and R4BCoreSchemaProvider (2-line fix) P1 Fix Required (test code)SmartSearchTests.cs line 108: use real FirelyFhirPathProvider instead of mock Full report: docs/features/sdk-migration/reports/ignixa-test-readiness-report.md |
[DO NOT REVIEW] Draft PR to trigger CI E2E tests for Ignixa SDK integration failure analysis. Local results: 1638 unit tests pass, integration non-Smart tests pass, 107 Smart failures due to known test setup bug. E2E needs CI. See docs/features/sdk-migration/reports/ignixa-test-readiness-report.md for full analysis.