How to connect to Spark Cluster?

[krishnaarava]

Hi,

Can you help with connecting to the Spark Cluster using Scala API?
(from outside the docker containers)

Below is my scala conf that I have

val SparkMasterHost = "127.0.0.1"
val CassandraHost = "127.0.0.1"

// Tell Spark the address of one Cassandra node:
val conf = new SparkConf(true)
.set("spark.cassandra.connection.host", CassandraHost)
.set("spark.cassandra.auth.username", "cassandra")
.set("spark.cassandra.auth.password", "cassandra")
.set("spark.cleaner.ttl", "3600")
.setAppName(getClass.getSimpleName)

// Connect to the Spark cluster:
lazy val sc = new SparkContext("spark://127.0.0.1:7077", "test", conf)