How to do an internship in E3DA

-written by Lorenzo Marogna-

Hey yo, welcome to E3DA! You will surely have some fun here, you just need to work seriously and be professional. Understanding how to do an internship here is not an easy thing to do, there is probably a lot of stuff you’ve only heard of before and never used. So I decided to write something about it to make life easier for newcomers. It follows that, if this sucks and it’s useless, don’t be angry at me, you’ll just need to learn new stuff without “extra” help. I am writing what I learned during my internship, so my suggestions might also not apply to you, sorry!

Rule 1: Pray

This is mainly just for the meme (we like them down here), but sometimes it is necessary. Don’t worry, your supervisors will help you if you need them, but try to be as autonomous as possible! Bring a Bible with you, it might be useful to make experiments work.

Docker

Docker is a powerful tool for containerizing applications and bla bla bla just write Docker on Google and you’ll find out.

From my experience: this is what you need to know:

docker ps: shows list of running containers
docker build: creates the image for running a new container. Once you have an image, you can launch more than one container from it.

# Description: Build docker images given the folder of the dockerfile
docker build --no-cache -t $IMAGE_NAME . # -t is the name of the image, . is the path where to find the DOCKERFILE

In order to use docker build you’ll need to have a Dockerfile. Here is an example of the one that I was using. You should probably be able to keep it like that and it should be fine, just provide a requirements.txt for the pip install at line 26.

<aside> ⚠️ IMPORTANT! You need to change USER_ID and USER_GROUP_ID. You can find out what values to use with the command id. Just use the IDs that you’ll get. It’s as easy as that.

</aside>

Dockerfile

docker run: launches a container starting from an image

#!/bin/bash

# Define variables

containername="default_container_name_to_change" # default name

# It is usually necessary to provide the name of the container
if [ "$#" -eq 1 ]; then
echo "one argument is provided"
containername=$1
fi

# Run docker
docker run --name $containername --gpus all --rm -it --shm-size=32gb -e "DOCKER_CONTAINER_NAME=$containername" -v  $PWD/volume:/workspace/ -v /raid/home/e3da/datasets/vision/$DATASETNAME/:/workspace/data/datasets/vision/$DATASETNAME/ $IMAGE_NAME bash

-v flag is used to mount stuff and synchronize between what’s outside and inside the container (remember that outside means the remote machine, not your laptop). For example:

$PWD/volume:/workspace/

synchronizes the content inside volume in the remote machine inside the workspace folder of the container. That is, the container will be created with a folder workspace that contains everything you put into the volume folder.