Skip to content

README.md

Latest commit

 

History

History
97 lines (69 loc) · 2.4 KB

README.md

File metadata and controls

97 lines (69 loc) · 2.4 KB
Raw

Data Analytics

Use Dockerfile to build image for runc and runsc.

Note: Large data files are stored in GitHub Releases and downloaded during Docker build. See LARGE_FILES.md for details.

Manual File Setup (Alternative)

If GitHub Releases download fails, manually place these files:

  1. Download from release: https://github.com/Anjali05/platform-workloads/releases/tag/dataAnalytics-v1.0.0

  2. Place files in these locations:

    cloud-workloads/dataAnalytics/data/wiki.gz          # 321MB compressed wiki data
cloud-workloads/dataAnalytics/files/mahout-examples-0.13.0-job.jar  # 77MB Mahout JAR

  3. Update Dockerfile to use local files:

    # Replace the GitHub download section with:
COPY data /data
COPY files/mahout-examples-0.13.0-job.jar ${MAHOUT_HOME}/.
RUN cd /data && gunzip wiki.gz

For runc and runsc

Start master:

docker run -itd --rm --cpuset-cpus=0-3 --memory=24g --name test mydataanalytics --master --master-ip=127.0.0.1

Start slave:

docker exec test /root/docker-entrypoint.py --slave --master-ip=127.0.0.1

Run workload:

docker exec test /root/benchmark.sh

For Firecracker (fc)

Building the Firecracker Image

Resize the base image to 16GB, then boot the VM and run:

# Install Java + Hadoop 2.10.2 (mirrors commons/hadoop/2.10.2/Dockerfile)
bash fc/hadoop_setup.sh

# Download Mahout 14.1, copy benchmark scripts, wiki data (/data/wiki),
# and categories (/user/categories)
bash fc/mahout_setup.sh

Copy the wiki dataset out of the CloudSuite data-analytics dataset container and transfer it into the VM at /data/wiki. Then copy the runtime files:

cp fc/hosts /etc/hosts
cp fc/benchmark.sh /root/benchmark.sh
chmod +x /root/benchmark.sh

Note: IPv6 must be enabled on the host and microVM.

Running the Workload

Copy fc/hosts to /etc/hosts in the microVM.

In Firecracker:

Start master:

/root/docker-entrypoint.py --master --master-ip=127.0.0.1

Start slave:

/root/docker-entrypoint.py --slave --master-ip=127.0.0.1

Start benchmark:

/root/benchmark.sh

Add to benchmark:

export JAVA_HOME=/usr/lib/jvm/java-11-openjdk-amd64
export PATH=$JAVA_HOME/bin:$PATH

Note: IPv6 must be enabled on the host and microVM.