Install-pytorch-and-detectron-on-ubuntu-16-04

1. dual OS:windows 10 and ubuntu 16.04

(1) install windows 10 firstly

(2) install ubuntu 16.04.03

disk partition:/boot,500MB,logic,ext4; /, 1000000MB, logic,ext4; swap,logic, swap, the left

• note:
  If you want to uninstall or reinstall your ubuntu in dual OS platform, please use easyUEFI kit delete ubuntu os in windows platform and then delete ubuntu volumes using disk management.

2. some configurations in ubuntu

2.1 change pawss word.

ubuntu ask you using longer password when you install it. Maybe you want to change a shorter pass word at the sake of convinence.

sudo passwd username

2.2 adjust the of grub.

The font of grub will be very small if your display is a high definition screen. You have two choices to change it. One is to unsample the grub resolution, the other is using bigger grub font. I think the later is better.

(1) unsample the grub resolution.

• to watch which resolutions are supported:press �c� key when grub menu is displayed; videoinfo; record which resolutions are supported in grub and select what you want.
• edit “ /etc/default/grub” line : “ #GRUB_GFXMODE=640x480 “ to match ( removing the starting comment character) . 800 * 600 is OK
• sudo update-grub

(2) change grub font size.

• sudo grub-mkfont --output=/boot/grub/fonts/DejaVuSansMono36.pf2 --size=36 /usr/share/fonts/truetype/dejavu/DejaVuSansMono.ttf
• sudo gedit etc/default/grub, add code below on the last line

# More readable font on high dpi screen, generated with
# sudo grub-mkfont --output=/boot/grub/fonts/DejaVuSansMono36.pf2 \
#    --size=36 /usr/share/fonts/truetype/dejavu/DejaVuSansMono.ttf
GRUB_FONT=/boot/grub/fonts/DejaVuSansMono36.pf2

• sudo update-grub

2.3 Change console font size:With HiDPI display you may find your console font size is too small.

To change it:

• $ sudo dpkg-reconfigure console-setup
  Encoding: UTF-8
  Character set: . Combined - Latin; Slavic Cyrillic; Greek
  Font: Terminus
  Font size: 14x28 (framebuffer only) or the even bigger

###2.4 Install google pinyin in Ubuntu 16.04

• Command line: sudo apt-get install fcitx-googlepinyin
• System settings -> Language support -> Keyboard input method system, change to fcitx.
• Log out log in
• At top right, click the penguin icon -> Text entry setting
• Click +
• Search �Google�, find �Google Pinyin (Fcitx)�
• Use �Ctrl+space� to switch between input methods

###2.5 update and upgrade ubuntu1604

sudo apt-get update
sudo apt-get upgrade

###2.6 Dual graphic card problem.
You often have two graphic cards, one is integrated and the other is standalone. You can designate use standalone nvidia graphic card in bios settings. But I don�t suggest to do it. Because your computer will often fan speed the the noise of fan will annoy you. In default, ubuntu uses integrated graphic card. To change it:

• Additional drivers. Using nvidia binary driver-version nvidia 384.130 from nvidia 384(proprietary, tested); apply; restart;
• nvidai-smi
• nvidia-settrings

• note: Don’t use the latest nvidia driver, e.g. nvidia 390.77. Some instruction about installing nvidia driver suggest to update nvidia driver by using the latest runfile, but I found it will lead that you have not PRIME Profiles item in your nvidia settings UI. So you can not select which card you want to use.

###2.7 install cuda and cudNN.
I think the latest cuda tool kit have some tiny problems about caffe2 and detectron. So I select the 8.0.61 cuda version and cudNN 6.0.21 which is also tested and suggested by the authors od detectron.

(a)Install cuda 8.0

• [ ] Download cuda 8.0.61_375.26 and its patch https://developer.nvidia.com/compute/cuda/8.0/Prod2/local_installers/cuda_8.0.61_375.26_linux-run
  https://developer.nvidia.com/compute/cuda/8.0/Prod2/patches/2/cuda_8.0.61.2_linux-run

• [ ] cd Downloads && chmod a+x cuda_8.0.61_375.26_linux.run

• sudo ./cuda_8.0.61_375.26_linux.run �no-opengl-libs
  accept installation,not install driver,yes install tool kit, for intstalling to default directory,yes for create soft link and not copy samples

• [ ] sudo gedit ~/.bashrc
  Add these lines at the bottom of the textexport.Some paths set for incoming installation of pytorch, detectron

  'PATH=/usr/local/cuda/bin${PATH:+:${PATH}}
  

  export PYTHONPATH=/usr/local:/home/lxn/pytorch/build${PYTHONPATH:+:${PYTHONPATH}}
  export LD_LIBRARY_PATH=/usr/local/cuda/lib64:/usr/local/lib:/home/lxn/pytorch/torch/lib${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

  export COCOAPI=/home/lxn/cocoapi
  export DETECTRON=/home/lxn/detectron'
  

• [ ] source ~/.bashrc

(b)install cuda 8.0 patch 2

• cd ~/Downloads && chmod a+x cuda_8.0.61.2_linux.run
• sudo ./cuda_8.0.61.2_lin ux.run

(c) install cudNN

• downloaded cudnn 6.0, https://developer.nvidia.com/compute/machine-learning/cudnn/secure/v6/prod/8.0_20170307/cudnn-8.0-linux-x64-v6.0-tgz
• cd ~/Downloads && sudo tar -xzf cudnn-8.0-linux-x64-v6.0.tgz -C /usr/local
• sudo ldconfig

(d) validate the installation:

• check nvcc version:nvcc -V, you can get �Cuda compilation tools, release 8.0, V8.0.61�
• check cuda version:cat /usr/local/cuda/version.txt, you will get message �CUDA Version 8.0.61,CUDA Patch Version 8.0.61.2�
• check cudNN version:cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2
• CUDA Sample Testing

  • testing deviceQuery:
    ? cd /usr/local/cuda-8.0/samples/1_Utilities/deviceQuery
    ? sudo make
    ? ./deviceQuery
  • testing bandwidthTest:
    ? cd ../bandwidthTest
    ? sudo make
    ? ./bandwidthTest

If both of the tests return Result = PASS,it shows cuda installation succeed.

2.8 install pytorch**

• [ ] Install Dependencies

  sudo apt-get update
  sudo apt-get install -y --no-install-recommends \
    build-essential \
    cmake \
    git \
    libgoogle-glog-dev \
    libgtest-dev \
    libiomp-dev \
    libleveldb-dev \
    liblmdb-dev \
    libopencv-dev \
    libopenmpi-dev \
    libsnappy-dev \
    libprotobuf-dev \
    openmpi-bin \
    openmpi-doc \
    protobuf-compiler \
    python-dev \
    python-pip                          
  

  pip install –user \

  future \
  numpy \
  protobuf
  

  sudo apt-get install -y –no-install-recommends libgflags-dev

• [ ] git clone https://github.com/pytorch/pytorch.git && cd pytorch

• git submodule update –init �recursive
  note: On current(August 29,2018), I found the latest version of pytorch will incur some troubles in compile process, So I used a historic version v0.4.1
  ? git tag(show all the tags of pytorch)
  ? git show v0.4.1(get the commit:a24163a95edb193ff7b06e98cd69bf7cfd4c0d2f)
  ? git reset –hard a24163a
  � FULL_CAFFE2=1 python setup.py install –user

  • note:
    The steps are copied from the official website of caffe2.But in practice, there are some tiny and noisy problems when you set up caffe2 at last. It maybe involve belows:
    (a) when pip install –user future, maybe need:
    ? pip install –upgrade –user setuptools
    ? pip install –user wheel
    (b) when build caffe2, maybe need:
    ? pip install –user pyyaml typing
    ? pip install –user hypothesis==3.40.0 (the version is important)
    ? pip install –user pydot graphviz

###2.9 install COCO API

$COCOAPI=/path/to/clone/cocoapi

git clone https://github.com/cocodataset/cocoapi.git $COCOAPI
cd $COCOAPI/PythonAPI

Install into global site-packages

make install

Alternatively, if you do not have permissions or prefer
to install the COCO API into global site-packages

python2 setup.py install --user

• note: The last step may fails. The main reason is the version of pip and location. You should:
  � pip install –user Cython
  � sudo apt-get remove python-pip
  � pip install –user –upgrade pip
  � pip install –user pip
  � pip install –upgrade –user setuptools
  delete matplotlib and reinstall matplotlib==2.2.3
  � sudo apt-get remove –purge python-matplotlib
  � pip install numpy �user
  � pip install scipy �user
  � pip install matplotlib==2.2.3 �user

###2.10 install detectron

• [ ] Clone the Detectron repository:

  DETECTRON=/path/to/clone/detectron

  git clone https://github.com/facebookresearch/detectron $DETECTRON

• [ ] Install Python dependencies:

  pip install -r $DETECTRON/requirements.txt

• [ ] Set up Python modules:

  cd $DETECTRON && make

• [ ] Check that Detectron tests pass (e.g. for SpatialNarrowAsOp test):

  python2 $DETECTRON/detectron/tests/test_spatial_narrow_as_op.py

• [ ] infer

  python2 tools/infer_simple.py \

  --cfg configs/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml \
  --output-dir /tmp/detectron-visualizations \
  --image-ext jpg \
  --wts https://s3-us-west-2.amazonaws.com/detectron/35861858/12_2017_baselines/e2e_mask_rcnn_R-101-FPN_2x.yaml.02_32_51.SgT4y1cO/output/train/coco_2014_train:coco_2014_valminusminival/generalized_rcnn/model_final.pkl \
  demo
  

• note: 4G GRAM sometimes is insufficient, You can use command to show the usage of GRAM:watch -n 1 nvidia-smi. But The best way is run infer process in tty.