In the previous post I have presented some of the first impressions for the Friendlyarm‘s NanoPi NEO Air board.

This post will focus on qualitative measures, specifically we will see how the board performs in a series of tests and how those results compare with various Raspberry Pi boards. For benchmark we will use the result published by Element 14 for three Raspberry Pi boards (model 3, model 2 and model 1 B+). We will try to stay as close as possible to the test conditions presented in that article.

Preparation

We will put the board through a lot of stress tests and it is expected that the processor will get hot. Fortunately the guys at Friendlyarm have already an answer for this: a heatsink that is very cool (he, he) and cheap. You should certainly consider it when buying your Nano. Here is a picture of the kit that comes with the heatsink:

img_1743.jpg

And here is a picture of the heatsink installed:

img_1744.jpg

The instructions are very easy to follow and the assembly is a breeze. The only thing that I have changed were the M2.5 screws that came with the sink with some M2 x 10mm screws because the original ones were too long for my taste and the M2.5 nuts were a little in the way of the antenna.

(A small parenthesis here. You can see I decided to install on the NanoPi board female headers instead of the male headers that come with the board. The reason for this is that the female headers can be used as support for the antenna. As you can see in the picture this makes the board a very compact ensemble. The only drawback is that the mounting holes are a little harder to use to mount the whole assembly to a robot as the heatsink is only hold in place by the adhesive of the thermal paste until all the screws are put in place. The option I have chosen instead is to design the add-on board a little longer than the NanoPi and have additional holes on that board that will be used to fix the fully assembled board, heatsink, antenna and add-on board to the robot. But more on this later.)

Secondly we will make sure that we provide enough power to the board by using a good power supply (2.1A). I have also ordered a couple of USB voltage testers that can be added easily on the power chain to monitor the current:

s-l1600

CPU Info and lscpu

To keep in line with the Element 14 article we will start with the CPU info report. We don’t have cpuinfo on Ubuntu, but we can

cat /proc/cpuinfo

that gives the same details. Here is how it compares with the Raspberry Pis:

Measure Raspberry Pi 3 Raspberry Pi 2 Raspberry Pi B+ NanoPi NEO
processor 0 0 0 0
model name ARMv7 Processor rev 4 (v7l) ARMv7 Processor rev 4 (v7l) ARMv6-compatible processor rev 7 (v6l) ARMv7 Processor rev 5 (v7l)
BogoMIPS 76.80 57.60 2.00 2275.81
features half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm crc32 half thumb fastmult vfp edsp neon vfpv3 tls vfpv4 idiva idivt vfpd32 lpae evtstrm half thumb fastmult vfp edsp java tls swp half thumb fastmult vfp edsp thumbee neon vfpv3 tls vfpv4 idiva idivt
CPU implementer 0x41 0x41 0x41 0x41
CPU architecture 7 7 7 7
CPU variant 0 0 0 0
CPU part 0xd03 0xc07 0xb76 0xc07
CPU revision 4 5 7 5
Hardware BCM2709 BCM2709 BCM2708 sun8i
Revision a02082 a01041 0010 0000

The BogoMIPS reported by the NanoPi seems suspicious and I assume there is some bug in the kernel code. It’s interesting to compare the features – the H3 in NanoPi seems to have a few missing compared to the BCM2709 in Raspberry Pi 3, but there are also a couple missing the other way around.

Running lscpu we get the following:

Measure Raspberry Pi 3 Raspberry Pi 2 Raspberry Pi B+ NanoPi NEO
Architecture armv7l armv7l armv6l armv7l
Byte Order Little Endian Little Endian Little Endian Little Endian
CPU(s) 4 4 1 4
On-line CPU(s) list 0-3 0-3 0 0-3
Thread(s) per core 1 1 1 1
Core(s) per socket 4 4 1 4
Socket(s) 1 1 1 1
Model name  ARMv7 Processor rev 4 (v7l)  ARMv7 Processor rev 5 (v7l) ARMv6-compatible processor rev 7 (v6l) n/a
CPU max MHz 1200 900 700 1200
CPU min MHz 600 600 700 480

Power Usage

I’m still waiting for the current measuring devices to be delivered. Once I receive them I will update the post with the readings for NanoPi.

SysBench

Now let’s do some proper testing… The first test executed was sysbench, specifically the calculation of prime numbers t0 20,000 with the following command:

sysbench --num-threads=4 --test=cpu --cpu-max-prime=20000 --validate run

Since we have only 4 cores we run with 4 threads and we will compare with the results on the Raspberry Pis. Here are the results:

Measure Raspberry Pi 3 Raspberry Pi 2 Raspberry Pi B+ NanoPi NEO
Total Time 119.47s 191.90s 1,321.49s 161.48s
min 47.69ms 76.38ms 412.94ms 62.71ms
avg 47.78ms 76.74ms 528.54ms 64.58ms
max 59.04ms 101.45ms 573.00ms 207.63s
difference min max 11.35ms 25.07ms 160.06ms 144.92ms

The performance of the NanoPi seems to be bang on between the Raspberry Pi 2 and 3 which is not bad considering the size of the board. One interesting thing to notice is the fact that the worst performing event (207.63ms) is way outside the average (64.58ms). That probably can be explained by the fact that the processor starts from a low energy mode (480MHz) and scales up during the running of the test. The rest of the processing is done at maximum speed – hence an average that is very close to the best performing event.

This behaviour is very obvious if we look at the system statistics that the NanoPi Monitor (a rebranded RPi-Monitor) shows. The statistics are accessible from the web browser at the IP address of the NanoPi port 8888:

NPI-Monitor

If we look at the Statistics tab and filter for the time when the test was run we have the following graphics:

graphic1

You can see the CPU clock going up sharply when the test started and staying there during the test then going down slowly once the test is completed. It’s also interesting to see the dynamics of the processor temperature. You can see it going up relatively fast initially and then tampering slightly. I might have to check the behaviour for a longer test to see the shape of that curve (will it go asymptotically towards 80, 90 or more degrees? and tapper naturally or will the CPU speed will be decreased to reduce the temperature?). What is even more interesting is that once the test is finished the temperature drops in a little more than 30 seconds from 68-69º to 55°. This is the impact of the heatsink and is remarkable considering it is a passive one.

nBench

I had to compile the nBench from source. Once done that I simply run it and here are the results. First the dump of the results of nBench:

BYTEmark* Native Mode Benchmark ver. 2 (10/95)
Index-split by Andrew D. Balsa (11/97)
Linux/Unix* port by Uwe F. Mayer (12/96,11/97)

TEST : Iterations/sec. : Old Index : New Index
 : : Pentium 90* : AMD K6/233*
--------------------:------------------:-------------:------------
NUMERIC SORT : 543.83 : 13.95 : 4.58
STRING SORT : 63.629 : 28.43 : 4.40
BITFIELD : 1.496e+08 : 25.66 : 5.36
FP EMULATION : 79.722 : 38.25 : 8.83
FOURIER : 6514.4 : 7.41 : 4.16
ASSIGNMENT : 10.123 : 38.52 : 9.99
IDEA : 2095.4 : 32.05 : 9.52
HUFFMAN : 924.2 : 25.63 : 8.18
NEURAL NET : 8.455 : 13.58 : 5.71
LU DECOMPOSITION : 310.97 : 16.11 : 11.63
==========================ORIGINAL BYTEMARK RESULTS======================
INTEGER INDEX : 27.637
FLOATING-POINT INDEX: 11.747
Baseline (MSDOS*) : Pentium* 90, 256 KB L2-cache, Watcom* compiler 10.0
==============================LINUX DATA BELOW===========================
CPU : 4 CPU
L2 Cache :
OS : Linux 3.4.39-h3
C compiler : gcc version 5.2.1 20151010 (Ubuntu 5.2.1-22ubuntu2)
libc : libc-2.21.so
MEMORY INDEX : 6.177
INTEGER INDEX : 7.491
FLOATING-POINT INDEX: 6.515
Baseline (LINUX) : AMD K6/233*, 512 KB L2-cache, gcc 2.7.2.3, libc-5.4.38
* Trademarks are property of their respective holder.

And here is a comparison of the numbers presented in the Element 14 post with my results for NanoPi NEO Air:

Measure Raspberry Pi 3 Raspberry Pi 2 Raspberry Pi B+ NanoPi NEO
memory index 7.105 4.186 2.501 6.177
integer index 8.976 5.812 3.208 7.491
floating point index 7.601 4.526 1.884 6.515

Again the numbers are between the ones of Raspberry Pi 2 and 3 but they seem to be significantly closer to 3 than to 2.

The following graphic shows the CPU frequency and temperature during the test:

graphic2

The test started around 19:00:38 and finished around 19:05:18. You can see the processor hardly goes over 50° which indicates that the sysbench is more demanding than nBench.

Roy’s Benchmark Collection

The article on Element 14 is a little cryptic about how to run these tests. There is a forum thread created by Roy that includes a link to the full pre-complied test for Raspberry Pi. Once downloaded and unzipped you need to set the execution flag for the tests that you need to run, for example:

chmode a+x memspeedPiA6

And then run it:

sudo ./memspeedPiA6

Here are the results of the two tests that are shown in the Element 14 post so that we can compare.

First memspeedPiA6:

test_res1

The specs are again between the Raspberry Pi 2 and 3 with performance closer to the version 3 than 2.

For NEON Speed Test v1.0 the results were:

test_res2

These numbers are better than the Raspberry Pi 2 but this time they are significantly inferior to the Raspberry Pi 3:

pastedImage_14

This might be an indication that the H3 implementation of NEON is not as good as the one done by Broadcom.

Finally we run the Linpack Single Precision Benchmark. The result for NanoPi NEO Air was 406.60MFLOPS. That compares to the 299.93 MFLOPS of Raspberry Pi 2 and 462.07 MFLOPS of Raspberry Pi 3. Again a result between Pi2 and 3 but close to the last one.

Conclusion

The results presented above present a very interesting picture: the NanoPi is not as good as the Raspberry Pi 3 (I was not expecting that anyway) but it is clearly better than a Raspberry Pi 2. In fact, if we were to represent the difference between the Pi2 and 3, the Nano will be somewhere 2/3 towards the Pi3 on average.

That is a remarkable result considering the price and the size of this board and really places this tiny board at the top of my choices to use in robots.

Advertisements