digging deep
Ran the shell script to download the binary files and convert them to HDF5. First the download failed because the wget utility wasn’t installed. I got wget for El Capitan from rudix.org, then ran the script again, download successful. Then ran convert.jl and this worked too. Two HDF5 files have been created, named train and text. I can open either using h5dump:
Got the shell script from here: https://github.com/pluskid/Mocha.jl/blob/master/examples/cifar10/get-cifar10.sh
It will download the binary files from the CIFAR-10 dataset and convert them to HDF5 format which is readable by Mocha ( this script calls convert.jl to do the conversion).
Resolved the HDF5 error by pushing the path line to Julia config file (as suggested in the HDF5 installation docs).
push!(Libdl.DL_LOAD_PATH, “/opt/local/lib”)
Pkg.build(“HDF5”)
So the problem before was that HDF5 was installed but Julia wasn’t seeing it.
To recap, got this error on running the Julia convert script:
UndefVarError: h5open not defined
in macro expansion; at ./REPL[9]:2 [inlined]
So the HDF5 module is not correctly installed, although I’d assumed it was when I ran convert.jl.
The top line of output from the compiler confirms that:
ERROR: LoadError: LoadError: HDF5 not properly installed. Please run Pkg.build(“HDF5”)
So reinstalled HDF5 as recommended:
julia> Pkg.build(“HDF5”)
Getting build errors…. and the compiler suggests:
— build the package(s) and all dependencies with `Pkg.build(“HDF5”)`
– build a single package by running its `deps/build.jl` script
But I’ve already tried the package manager build twice without success, so next I’ll try cloning the HDF5 code from Github.
The binary files of the CIFAR-10 dataset need to be converted to HDF5 as that is the format recognised by Mocha. To do this, I will use the Julia script convert.jl provided by pluskid. First, in Julia used the package manager to install HDF5: Pkg.add(“HDF5”). Then ran the conversion script. I get the following error:
UndefVarError: h5open not defined
in macro expansion; at ./REPL[9]:2 [inlined]
in anonymous at ./<missing>:?
Downloaded the Docker image from BVLC’s repository. First time I tried, the pull failed with a Docker i/o error: “failed to register layer” Then reset Docker to factory defaults. Then the pull worked: dowboaded newer image for bvlc/caffee: cpu.
I’m going to install Caffee on a system that doesn’t have a GPU. This MacBook Air 2014 only has Intel integrated graphics… no dedicated graphics chip. Is it possible to install and run Caffee without a GPU? Yes, according to their docs. Caffee can be configured to use only the CPU. Uncomment CPU_ONLY := 1 in MakeFile.config to get “cold-brewed” Caffee.
Of course, the deep learning algorithms required for image classification would run much faster with GPU support, as the parallel architecture of a GPU is ideally suited to those algorithms. But they can still run on the CPU alone. So I’m going to try the deep learning tutorial without a GPU. Something may be learned by starting from the minimum and working up.
I found out yesterday that I’d have to install Caffee first before I could start the demo in Mocha. Caffee is a CNN (Convolutional Neural Network) developed by Berkely Vison and Learning Center. The tutorial I’m following (at mochajl.readthedocs.io) requires using Caffee’s API to export the Caffee model parameters to HDF5 format. After that, the exported file can be input to Mocha. To sum up, I have to install Caffee first. That’s next…
Setting up Julia locally first, before trying the cloud environment. Running OS X, El Capitan. Python and iPython weee already installed. So I first installed Julia, then installed IJulia.jl from the Julia REPL (Read Eval Print Loop). IJulia is an interface between Julia and Jupyter notebooks. Then installed Images.jl, also from the Julia REPL. Then Gadfly, a visualization tool written in Julia. Will continue tomorrow night.
Since my last post, I discovered the Julia language. Like Lua, it is a high-performance programming language ideally suited to numeric analysis. Depending on how I get on with Julia, I may decide to run Mocha (a machine learning library for Julia) on an AWS instance instead of Torch.