For upgrading the image, we will be using a bin file of the existing image already developed. Let us download a .bin file to the directory we have created.

Before that, let us install wget package. For doing so, execute the following command
yum install wget -y

Now, navigate to the Source directory and download the bin file by executing the following command(Due to space constraint in Katacoda we have uploaded the required files into dropbox)
cd ~/Source
wget https://www.dropbox.com/s/r4m3bba2qtueu5q/bdcatalog-centos7-bluedata-spark221-1.0.bin.tar

Extract the content of the tar file by executing the following command
tar xvf bdcatalog-centos7-bluedata-spark221-1.0.bin.tar

Now, Make the .bin file executable by executing the following command
chmod +x bdcatalog-centos7-bluedata-spark221-1.0.bin

In this step we will extract the contents of the .bin file. For this execute the following command.
./bdcatalog-centos7-bluedata-spark221-1.0.bin --payload

This will create two files in the directory. (to view the files execute the following command)
ls

Untar the payload.tar file by executing the following command
tar xvf payload.tar

This will create new directories. (to view the files execute the following command)
ls

Remove the bin and tar file to save space on the device
rm -rf bdcatalog-centos7-bluedata-spark221-1.0.bin
rm -rf bdcatalog-centos7-bluedata-spark221-1.0.bin.tar

Navigate to the new directory, and untar the file by executing the following command
cd bdcatalog-centos7-bluedata-spark221-1.0
tar xvf bdcatalog-centos7-bluedata-spark221-1.0-src.tgz

Now navigate to the directory created after the above step
cd bdcatalog-centos7-bluedata-spark221-1.0-src

Copy the content of this directory to the directory you created initially
cp -R * ~/Source/Spark

Now, to upgrade an existing image to a new one, we should make changes in some of the files like:

Now, we will make changes in .wb file first.

.wb file contains a series of App Workbench commands that control the creation of the Catalog Image
Please review the link before proceeding for wb file: Link

For your reference, we have already created a ready .wb file in the ~/test directory. We will copy that file into the Spark directory using the following command:
cd ~/Source/Spark

Delete the existing .wb file and json file before you copy the new .wb file by executing the below command
rm -rf spark221.wb spark221e1.json

Copy the new .wb file into Spark directory using the following command:
cp ~/test/spark_notebook.wb ~/Source/Spark

To view the content of the file, execute the following
cat ~/Source/Spark/spark_notebook.wb

The next step is to create a Dockerfile.

"A docker file is a text file that the Docker engine understands to automatically build an image by reading the file. The Dockerfile consists of all the commands a user would call to assemble the desired image."

Let’s go ahead and create a Dockerfile inside the newly created centos folder.
For your reference, we have already created a ready Dockerfile in the ~/test directory.We will copy that file into the centos folder using the following command:

Delete the existing Dockerfile by executing the below command:
rm -rf ~/Source/Spark/image/centos/Dockerfile

Copy the new Dockerfile into the centos folder using the following command:
cp ~/test/Dockerfile ~/Source/Spark/image/centos/

Copy the updatePath.sh into the centos folder using the following command:
cp ~/test/updatePath.sh ~/Source/Spark/image/centos/

Execute the following command to view the contents of the Dockerfile:
cat ~/Source/Spark/image/centos/Dockerfile

We need to add additional configuration files under appconfig directory, We already have a reference configuration files, To add them follow the below steps

Remove the existing appconfig folder from the Spark folder
rm -rf appconfig

Add the appconfig reference file using the below command from stored location(Due to space constraint in Katacoda we have uploaded the required files into dropbox)
wget https://www.dropbox.com/s/f8cjkwzz6qdov0d/appconfig.zip
yum install unzip -y

Unzip the file
unzip appconfig.zip

Check files under appconfig directory
ls appconfig

Make sure you see following files unders appconfig folder:

Now, for creating the bin file, let us execute the .wb file.
./spark_notebook.wb

After executing the command, wait for sometime as the bin file will take time to create!!!!

The newly built application package (or bundle) is saved in the deliverables directory.
cd deliverables
ls

Make it an executable using the below command
chmod +x bdcatalog-centos7-bluedata-spark231mjuphub7x-2.1.bin

Congratulations you have successfully upgraded spark with notebook image.

To make the new image appear in App Store Copy the bin file to /srv/bluedata/catalog

Go to App Store in the EPIC GUI and click on Refresh button to bring the image in to App Store.

Once the image appears on the App Store click on Install button to install the image.