DSBulk usage:

dsbulk <command> [options]

DSBulk supports the following three commands:

DSBulk has dozens of options. Here are some of the most common ones:

You can learn more about these and other options from the help page:

dsbulk help

Create the keyspace and four tables using the CQL shell:

cqlsh -e "

CREATE KEYSPACE killr_video
WITH replication = {
  'class': 'NetworkTopologyStrategy', 
  'DC-Houston': 1 };

USE killr_video;

CREATE TABLE users (
  id TEXT,
  gender TEXT,
  age INT,
  PRIMARY KEY ((id))
);

CREATE TABLE movies (
  id TEXT,
  title TEXT,
  year INT,
  duration INT,
  country TEXT,
  PRIMARY KEY ((id))
);

CREATE TABLE ratings_by_user (
  user_id TEXT,
  movie_id TEXT,
  rating INT,
  PRIMARY KEY ((user_id), movie_id)
);

CREATE TABLE ratings_by_movie (
  movie_id TEXT,
  user_id TEXT,
  rating INT,
  PRIMARY KEY ((movie_id), user_id)
);"

Our first task is to load user data from file users.csv with header fields user_id, gender and age into table users with columns id, gender and age.

Output the first five lines from the file:

head -n 5 users.csv

Verify that the table is empty:

cqlsh -e "SELECT * FROM killr_video.users LIMIT 5;"

Since the file field names and the table column names do no match exactly, we have to map fields to columns explicitly. There are many ways to do this as we demonstrate in the following examples.

Load data (name-to-name mapping):

dsbulk load -url users.csv       \
            -k killr_video       \
            -t users             \
            -header true         \
            -m "user_id=id,      \
                gender=gender,   \
                age=age"         \
            -logDir /tmp/logs

Load data (position-to-name mapping):

dsbulk load -url users.csv       \
            -k killr_video       \
            -t users             \
            -header true         \
            -m "0=id,            \
                1=gender,        \
                2=age"           \
            -logDir /tmp/logs

Load data (skip the file header and specify the column names):

dsbulk load -url users.csv       \
            -k killr_video       \
            -t users             \
            -header false        \
            -skipRecords 1       \
            -m "id, gender, age" \
            -logDir /tmp/logs

Output five rows from the table:

cqlsh -e "SELECT * FROM killr_video.users LIMIT 5;"

Next, load movie data from file movies.csv into table movies.

Output the first five lines from the file:

head -n 5 movies.csv

Verify that the table is empty:

cqlsh -e "SELECT * FROM killr_video.movies LIMIT 5;"

Load data:

Output five rows from the table:

cqlsh -e "SELECT * FROM killr_video.movies LIMIT 5;"

Let's load movie ratings from file ratings.csv into tables ratings_by_user and ratings_by_movie.

Output the first five lines from the file:

head -n 5 ratings.csv

Verify that the tables are empty:

cqlsh -e "SELECT * FROM killr_video.ratings_by_user LIMIT 5;"
cqlsh -e "SELECT * FROM killr_video.ratings_by_movie LIMIT 5;"

Notice that the file field names and the table column names match. We do not need to provide an explicit mapping this time.

Load data into table ratings_by_user:

dsbulk load -url ratings.csv      \
            -k killr_video        \
            -t ratings_by_user    \
            -header true          \
            -logDir /tmp/logs

Load data into table ratings_by_movie:

Output five rows from each table:

cqlsh -e "SELECT * FROM killr_video.ratings_by_user LIMIT 5;"
cqlsh -e "SELECT * FROM killr_video.ratings_by_movie LIMIT 5;"

Finally, take a look at how commands unload and count can be used to export data from Cassandra and compute simple row counts.

Unload all rows from table ratings_by_movie:

dsbulk unload -url all_ratings    \
              -k killr_video      \
              -t ratings_by_movie \
              -header true        \
              -logDir /tmp/logs

Unload rows from table ratings_by_movie using a query:

dsbulk unload -url m267_ratings   \
              -query "            \
SELECT *                          \
FROM killr_video.ratings_by_movie \
WHERE movie_id = 'm267'"          \
              -header true        \
              -logDir /tmp/logs

Check the resulting CSV files:

head -n 5 all_ratings/*
head -n 5 m267_ratings/*

Count all rows in table ratings_by_movie:

dsbulk count  -k killr_video      \
              -t ratings_by_movie \
              -logDir /tmp/logs

Count rows in table ratings_by_movie using a query:

dsbulk count  -query "            \
SELECT *                          \
FROM killr_video.ratings_by_movie \
WHERE movie_id = 'm267'"          \
              -logDir /tmp/logs

Here is a short quiz for you.