Sources

TextFile Source

Read a text file from HDFS, a local file system, or any Hadoop-supported file system URI. Use SparkContext.textFile method.

Configuration key	Default	Description
path	-	List of paths to read. Path can be a file or directory.
filters*	defaultHadoopFilter	List of path filters.

• Path filter is a simple function defined as type PathFilter = Path => Boolean. Available filter types you can find here.