Exploring Neural Networks, p. 2

(see part 1 here)

Once I fixed all my bugs in Batch Normalisation implementation and fine-tuned all parameters, I started getting reasonable results. In particular, it turned out that I needed to significantly (more than 10 times) increase weight decay ratio constant. I also had to modify learning rate scheduling so that it decays much faster, this makes sense, because Batch Normalisation is supposed to speed up learning. Eventually, the network:

3 channels ->  64 3x3 convolutions -> 3x3 maxpool -> BN -> ReLU
           -> 128 3x3 convolutions -> 2x2 maxpool -> BN -> ReLU
           -> 1024 to 1024 product ->                BN -> ReLU
           -> 1024 to  512 product ->                BN -> Sigmoid
           ->  512 to   10 product
           ->  SoftMax

has achieved 79% success rate on the test set.

I was interested in the advantage of using BN. To investigate it, I created another network, which is an identical clone of the one described above, but no Batch Normalisations are performed at all. Comparing the results of these two networks should express the gain introduced by using BN.

Read the rest of this entry »

Exploring Neural Networks, p. 1

As a final assignment on the Neural Networks course I took part in (University of Wrocław, Institute of Computer Science, winter2015/2016), I am tasked with designing, implementing and training a neural net that would classify CIFAR-10 images with some reasonable success rate. I am also encouraged to experiment with the network by implementing some of the recent inventions that may, in one way or another, improve my network’s performance. I will be sharing my results and observations here, in this post, and in some that will follow soon within the next two weeks.

The source code I am using for my experiments is available at github. The sources come with a number of utilities that simplify running them on our lab’s computers, which may come in handy if you are a fellow student peeking at my progress, but if you are not, then you should ignore all files except the ones within ./project directory.

Read the rest of this entry »

QEMU – main-loop: WARNING: I/O thread spun for 1000 iterations

When upgrading the virtual machine I use, I stumbled upon an issue where the guest OS would hang every time when performing any kind of heavy hard drive I/O. Qemu monitor would only display:

main-loop: WARNING: I/O thread spun for 1000 iterations

Some digging led me to the following workaround:

diff -u a/vl.c b/vl.c
--- a/vl.c	2015-11-20 01:45:00.179169442 +0100
+++ b/vl.c	2015-11-20 01:44:22.181778840 +0100
@@ -1914,6 +1914,7 @@
 #endif
     do {
         nonblocking = !kvm_enabled() && !xen_enabled() && last_io > 0;
+        nonblocking = 0;
 #ifdef CONFIG_PROFILER
         ti = profile_getclock();
 #endif

For explanation of the nature of this issue, read this discussion.

Current progress on AlgAudio

… or “what I’ve been working on for the past three months”.

So this summer I have participated in a programming internship at Audiovisual Technology Center – CeTA in Wrocław. CeTA is developing a number of very exciting projects, and the one I had the pleasure to work on is AlgAudio.

screenshot

(download links available below)

AlgAudio is a new signal processing framework that we’ve been developing from scratch. The user builds an audio processing network by placing “building blocks” of simple operations, connecting them together, configuring their parameters, and defining how the parameters should influence each other. The network works in real time, so any changes to the parameters are immediately reflected in the outputted audio. This makes AlgAudio a perfect tool for live performances.

Read the rest of this entry »

Sloped triangles tesselation

  • Model: Sloped triangles tessellation
  • Designed and folded by: Rafał Cieślak
    •  Inspired by Eric Joisel’s Hedgehog.
    • I have been later shown I was not the first to come up with such pattern [1] [2]. That’s not surprising, given how simple the molecule is.
  • Paper size: A4
  • Folding time: ~4h

Sloped Triangles TesselationSloped Triangles Tesselation

Sloped Triangles TesselationSloped Triangles Tesselation

Folded on 20 VI 2015

Origami japanese beetle

  • Model: Robert J. Lang’s Samurai Helmet Beetle
    This is actually a Japanese Rhinoceros Beetle.
  • Folded by: Rafał Cieślak
  • Paper size: 30cm (12″) square sheet
  • Model size: 12cm (4.7″) length
  • Paper type: Golden-white waxed-tissue-foil¹
  • Folding time: ~4h

Japanese beetle   Japanese beetle

Japanese beetle   Japanese beetle

Japanese beetle   Japanese beetle

¹) Details on the paper used are described here.

I folded the model on 25 april 2015.

Multi-OS gaming w/o dual-booting: Excelent graphics performance in a VM with VGA passthrough

Note: This articles is a technology/technique outline, not a detailed guide and not a how-to. It explains what is VGA passthrough, why you might be interested in it, and where to start.

Even with the current abundance of Linux native games (both indies and AAAs), with WINE reliably running almost any not-so-new software, many gamers who use Linux on a daily basis tend to switch to Windows for playing games. Regardless of one’s attitude towards non-free software, it has to be admitted that if you wish to try out some of the newest titles, you have no other choice than running them on a Windows installation. This is why so many gamers dual-boot: having installed two operating systems on the same machine and using Windows for playing games and Linux for virtually anything else, they limit their usage of Microsoft’s OS for gaming only. This popular technique seems handy – you get the luxury of using a Linux, and the gaming performance of Windows.

But dual-booting is annoying because of the need of reboot to switch your context. Need to IM your friend while playing? Save your game, shut down Windows, reboot to Linux, launch IM, reboot to Windows, load your game. Switching takes a long time, is inconvenient, and therefore the player may feel discouraged to do so.

What if you could run both operating systems at once? That’s nothing new, run a virtual machine in your Linux, install Windows within it, and voilà! But a virtual machine is no good for gaming, the performance will be utter cr terrible. Playing chess might work, but any 3D graphics won’t do because of the lack of hardware acceleration. The VM emulates a simple graphics adapter to display it’s output in a window of the host OS.

And that is where VGA passthrough comes in, and solves this issue.

Read the rest of this entry »

Origami butterfly

  • Model: Robert J. Lang’s Butterfly
  • Folded by: Rafał Cieślak
  • Paper size: 36cm (14″) square sheet
  • Model size: 15cm (6″) wing span
  • Paper type: Waxed-tissue-foil¹
  • Folding time: ~5h

Butterfly   Butterfly

Butterfly   Butterfly Read the rest of this entry »

C++11: std::threads managed by a designated class

Recently I have noticed an unobvious problem that may appear when using std::threads as class fields. I believe it is more than likely to meet if one is not careful enough when implementing C++ classes, due to it’s tricky nature. Also, its solution provides an elegant example of what has to be considered when working with threads in object-oriented C++, therefore I decided to share it.

Consider a scenario where we would like to implement a class that represents a particular thread activity. We would like it to:

  • start a new thread it manages when an instance is constructed
  • stop it when it is destructed

I will present the obvious implementation, explain the problem with it, and describe how to deal with it.

Read the rest of this entry »

Dynamic linker tricks: Using LD_PRELOAD to cheat, inject features and investigate programs

This post assumes some basic C skills.

Linux puts you in full control. This is not always seen from everyone’s perspective, but a power user loves to be in control. I’m going to show you a basic trick that lets you heavily influence the behavior of most applications, which is not only fun, but also, at times, useful.

A motivational example

Let us begin with a simple example. Fun first, science later.

#include <stdio.h>
#include <stdlib.h>
#include <time.h>

int main(){
  srand(time(NULL));
  int i = 10;
  while(i--) printf("%d\n",rand()%100);
  return 0;
}

Simple enough, I believe. I compiled it with no special flags, just

gcc random_num.c -o random_num

I hope the resulting output is obvious – ten randomly selected numbers 0-99, hopefully different each time you run this program.

Now let’s pretend we don’t really have the source of this executable. Either delete the source file, or move it somewhere – we won’t need it. We will significantly modify this programs behavior, yet without touching it’s source code nor recompiling it.

For this, lets create another simple C file:

int rand(){
    return 42; //the most random number in the universe
}

We’ll compile it into a shared library.

gcc -shared -fPIC unrandom.c -o unrandom.so

So what we have now is an application that outputs some random data, and a custom library, which implements the rand() function as a constant value of 42.  Now… just run random_num this way, and watch the result:

LD_PRELOAD=$PWD/unrandom.so ./random_nums

If you are lazy and did not do it yourself (and somehow fail to guess what might have happened), I’ll let you know – the output consists of ten 42’s.

Read the rest of this entry »