How to build/package Scala Projects using SBT and run as Apache Spark applications

Simple Build Tool (SBT) is an open source build tool for Scala and Java projects, similar to Java's Maven or Ant. Its main features are: native support for compiling Scala code and integrating with many Scala frameworks. sbt is the de facto build tool in the Scala community.

Getting started with a simple SBT project :

Pre-requisites: You could download and install listed packages:
1) sbt from: http://www.scala-sbt.org/    or  git clone https://github.com/sbt/sbt.git
2) scala from https://www.scala-lang.org/download/all.html
3) Apache spark from  http://spark.apache.org/downloads.html

   Typical sbt project will have following directory structure.

step 1:  create the folder scala_project :
             mkdir Scala_project
Step 2: cd  scala_project

Step 3 : mkdir -p project src/{main,test}/{scala,java,resources}

Step 4 : show the directory structure

Step 5 : create build.sbt file  at the root of the folder  i.e  Scala_project/

Step 6:  Create project/build.properties file
             echo "sbt.version=0.13.13" >project/build.properties

Step 7: Create a scala program "hello.scala"   at /src/main/scala directory

Step 8: You can directly compile this file using scalac  without sbt tool
             scalac hello.scala

Step 9:   Go to your projects root directory  i.e /home/spb/Scala_project  and run following commands

1) sbt clean - delete all previously generated files in target directory
-------------------------------------------------------------------------------------------
spb@spb-VirtualBox:~/Scala_project$ pwd
/home/spb/Scala_project
spb@spb-VirtualBox:~/Scala_

project$ ls
build.sbt  project  src  target
spb@spb-VirtualBox:~/Scala_project$ sbt clean
[info] Loading project definition from /home/spb/Scala_project/project
[info] Set current project to SPB_Hello world (in build file:/home/spb/Scala_project/)
[success] Total time: 0 s, completed 5 Mar, 2017 1:26:47 PM
spb@spb-VirtualBox:~/Scala_project$

----------------------------------------------------------------------------------------------

2) sbt update -Updates external dependencies.

 spb@spb-VirtualBox:~/Scala_project$ sbt update
[info] Loading project definition from /home/spb/Scala_project/

project
[info] Set current project to SPB_Hello world (in build file:/home/spb/Scala_project/)
[info] Updating {file:/home/spb/Scala_project/}scala_project...
[info] Resolving jline#jline;2.12.1 ...
[info] Done updating.
[success] Total time: 1 s, completed 5 Mar, 2017 1:26:59 PM
spb@spb-VirtualBox:~/Scala_project$

 -----------------------------------------------------------------------------------------------
3) sbt compile  -Compiles source code files that are in src/main/scala, src/main/java and root dir of project

----------------------------------------------------------------------------------------
spb@spb-VirtualBox:~/Scala_

project$ sbt compile
[info] Loading project definition from /home/spb/Scala_project/project
[info] Set current project to SPB_Hello world (in build file:/home/spb/Scala_project/)
[info] Compiling 1 Scala source to /home/spb/Scala_project/target/scala-2.11/classes...
[success] Total time: 5 s, completed 5 Mar, 2017 1:27:26 PM
spb@spb-VirtualBox:~/Scala_project$ 

--------------------------------------------------------------------------------------

4) sbt run -Compiles your code, and runs the main class from your project, in the same JVM as SBT. If your project has multiple main methods (or objects that extend App), you’ll be prompted to select one to run.
----------------------------------------------------------------------------------------
 spb@spb-VirtualBox:~/Scala_

project$ sbt run
[info] Loading project definition from /home/spb/Scala_project/project
[info] Set current project to SPB_Hello world (in build file:/home/spb/Scala_project/)
[info] Running HelloWorld
Hello world and welcome to scala
[success] Total time: 1 s, completed 5 Mar, 2017 1:27:55 PM
spb@spb-VirtualBox:~/Scala_project$ 

--------------------------------------------------------------------------------------

5) sbt package -Creates a JAR file (or WAR file for web projects) containing the files in src/main/scala, src/main/java, and resources in src/main/resources.
----------------------------------------------------------------------------------------

spb@spb-VirtualBox:~/Scala_project$ sbt package
[info] Loading project definition from /home/spb/Scala_project/project
[info] Set current project to SPB_Hello world (in build file:/home/spb/Scala_project/)
[info] Packaging /home/spb/Scala_project/target/scala-2.11/spb_hello-world_2.11-1.0.jar ...
[info] Done packaging.
[success] Total time: 0 s, completed 5 Mar, 2017 1:27:41 PM
spb@spb-VirtualBox:~/Scala_project$
---------------------------------------------------------------------------------------

Check your  target directory  for jar file created .

 
Now you can submit a job to Apache spark framework  to run application as shown here:

 ---------------------------------------------------------------------------------

spb@spb-VirtualBox:~/Scala_

project/target/scala-2.11$ spark-submit  --class  HelloWorld spb_hello-world_2.11-1.0.jar
Hello world and welcome to scalaspb@spb-VirtualBox:~/Scala_project/target/scala-2.11$

----------------------------------------------------------------------------------- 

Additionally , you could also make use of sbt console that Compiles the source code files in the project, puts them on the classpath, and starts the Scala interpreter (REPL). SBT has some interesting features that come in handy during development, such as starting a Scala REPL with project classes and dependencies on the classpath, continuous compilation and testing with triggered execution, and much more.

                                                                                ---THE END---