Skip to content

feat: add parquet reader - #145

Closed
georgeRobertson wants to merge 2 commits into
release_v010from
feature/gr-ndit-1761-add_parquet_reader
Closed

feat: add parquet reader#145
georgeRobertson wants to merge 2 commits into
release_v010from
feature/gr-ndit-1761-add_parquet_reader

Merge branch 'release_v010' of https://github.com/NHSDigital/data-val…

4bc4295
Select commit
Loading
Failed to load commit list.
SonarQubeCloud / SonarCloud Code Analysis failed Nov 3, 2026 in 28s

Quality Gate failed

Failed conditions
4.2% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

Annotations

Check warning on line 61 in src/dve/core_engine/backends/implementations/duckdb/readers/parquet.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Complete the task associated to this "TODO" comment. 

See more on https://sonarcloud.io/project/issues?id=NHSDigital_data-validation-engine&issues=AaGiQYlFsrTDYnXcR77e&open=AaGiQYlFsrTDYnXcR77e&pullRequest=202

Check warning on line 62 in src/dve/core_engine/backends/implementations/duckdb/readers/parquet.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Complete the task associated to this "TODO" comment. 

See more on https://sonarcloud.io/project/issues?id=NHSDigital_data-validation-engine&issues=AaGiQYlFsrTDYnXcR77f&open=AaGiQYlFsrTDYnXcR77f&pullRequest=202

Check warning on line 152 in src/dve/core_engine/backends/implementations/spark/contract.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Replace "list(...)[0]" with "next(iter(...))" to avoid materializing the entire iterable. 

See more on https://sonarcloud.io/project/issues?id=NHSDigital_data-validation-engine&issues=AaGiQYvUsrTDYnXcR77i&open=AaGiQYvUsrTDYnXcR77i&pullRequest=202

Check warning on line 52 in src/dve/core_engine/backends/implementations/spark/readers/parquet.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Specify both "master" and "appName" parameters to initialize this SparkSession. 

See more on https://sonarcloud.io/project/issues?id=NHSDigital_data-validation-engine&issues=AaGiQYtQsrTDYnXcR77h&open=AaGiQYtQsrTDYnXcR77h&pullRequest=202

Check warning on line 168 in src/dve/core_engine/backends/implementations/duckdb/contract.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Replace "list(...)[0]" with "next(iter(...))" to avoid materializing the entire iterable. 

See more on https://sonarcloud.io/project/issues?id=NHSDigital_data-validation-engine&issues=AaGiQYtPsrTDYnXcR77g&open=AaGiQYtPsrTDYnXcR77g&pullRequest=202

Check warning on line 92 in src/dve/core_engine/backends/readers/utilities.py

See this annotation in the file changed.

@sonarqubecloud sonarqubecloud / SonarCloud Code Analysis

Complete the task associated to this "TODO" comment. 

See more on https://sonarcloud.io/project/issues?id=NHSDigital_data-validation-engine&issues=AaGiQYvbsrTDYnXcR77j&open=AaGiQYvbsrTDYnXcR77j&pullRequest=202
View more details on SonarQubeCloud