Name	Name	Last commit message	Last commit date
parent directory ..
notebooks	notebooks
pyspark-3.x	pyspark-3.x
pyspark-4.x	pyspark-4.x
scala	scala
README.md	README.md
compose.spark-3.5-standalone.yaml	compose.spark-3.5-standalone.yaml
compose.spark-4.2-standalone.yaml	compose.spark-4.2-standalone.yaml
compose.yaml	compose.yaml
fhv_zones_gcs.py	fhv_zones_gcs.py
spark-3.5-standalone.conf	spark-3.5-standalone.conf
spark-4.2-standalone.conf	spark-4.2-standalone.conf

Name

Last commit message

Last commit date

Batch processing with Spark

Getting Started

docker compose up -d

Refer to PySpark 4.x for how to run the pipeline

docker compose -f compose.spark-3.5-standalone.yaml up -d

Refer to PySpark 3.5 for how to run the pipeline