korki’s corner

First of all I suggest to download from codesourcery the mips toolchain and do the thing manually. Hopefully sometime ubiquity will provide its users with a decent SDK. If you don’t take my word do the following. Pretty soon you will do the first

 Note that this is NOT COMPLETE NOR CORRECT.  I WILL UPDATE AGAIN, STAY TUNED.

• First of all download airocrack-ng and airos sdk.
• also download openssl, it will come handy (probably)
• if you have downloaded debian packaged airos sdk  and you don’t have debian derived distro, then unpack it using the following:
  • ar vx toolchain-mips-ls_0.1-1.deb (this is based on what I downloaded, check yours)
  • untargz the output (this is usually the large archive) : tar -zxvf data.tar.gz
  • update PATH environment variable to use mips compiler instead of the system’s (that is IF your distro comes with a compiler) export PATH=~/ubnt/opt/toolchain/mips_ls/bin/:$PATH (this is for my directory structure update yours accordingly)
• untargz airocrack distribution: tar -zxvf aircrack-ng-1.1.tar.gz (update accordingly)
• change directory to the newly created aircrack-ng-1.1: cd aircrack-ng-1.1
• execute there TOOL_PREFIX=mips-elf-linux-gnu- CFLAGS=”-O2 -march=24kc -EL -static -s” make if you are lucky and the so called airos SDK is a true SDK everything will compile normally

I am pretty sure it won’t.

SIDENOTE: Dear ubiquity toolchain developers: SDK MEANS SOFTWARE DEVELOPMENT KIT. THIS MEANS THAT IT SHOULD CONTAIN THE WHOLE FUCKIN’ LIST OF LIBRARIES THAT ARE NEEDED TO BUILD EXECUTABLES ON TOP OF AIRoS. NOT JUST A FUCKIN’ COMPILER FFS… This is a good time to go and download the codesourcery toolchain as mentioned in the beginning.

•  Due to these morons we need to probably build openssl, and statically link against airocrack-ng (duuuuuhhhh…)
• download openssl untar and enter directory
• edit Configure and add the following line somewhere in the middle of the definitions: (this is simply SAD)
  • “nanostation”, “mips-linux-gnu-gcc:-mabi=32  -DTERMIO -O3 -g -Wall -s -EL::-D_REENTRANT::-ldl:BN_LLONG RC2_CHAR RC4_INDEX DES_INT DES_UNROLL DES_RISC2::::::::::::dlfcn:linux-shared:-fPIC::.so.\$(SHLIB_MAJOR).\$(SHLIB_MINOR)”,
• ./Configure nanostation
• edit Makefile (with the following):
  • AR= mips-linux-ar $(ARFLAGS) r
  • RANLIB= mips-linux-ranlib
  • NM= mips-linux-nm
  • MAKEDEPPROG= mips-linux-gcc
• make (yeah right using the jobserver (ie -j5) is not a good idea…)
• and then compile and statically link againstour own ssl from airocrack.

Now it is the time to setup nfs in your host linux (there we are going to store the airodump capture, and from there we will launch airodump) [example tutorial].

• edit /etc/export and add something like:
  • /data/nanostation/ 10.140.4.0/255.255.255.0(async,rw,no_subtree_check)
• (re)start nfs server
• login to nanostation and mount it:
  • mkdir /tmp/lala
  • mount 10.140.4.1:/data/nanostation /tmp/lala
• execute airodump from there
  • cd /tmp/lala
  • ./airodump with your favorite options

Please do not try to crack anything from within the nanostation. You will waste precious machine cycles Instead do it from the host. Make good usage of the nfs man!

Obviously this is a synthetic question,

merely mimicking the known (recently resolved) dilemma “which came first, the chicken or the egg?“. The use of present tense is on purpose in order to set the stage in the current time. Of course this is not a paradox from the Godel’s incompleteness theorem (Any formal system that is interesting enough to formulate its own consistency can prove its own consistency iff it is inconsistent), yet it could be formulated into a formal system since compilers and their output can be deterministic. A compiler can be described mathematically as a FSM, or even as a Turing machine (I am not saying it is easy to implement it though), so it would be trivial to prove that a C compiler is Turing complete (a language, a compile etc is said to be Turing complete if and only if such system can simulate any single-taped Turing machine - Some people of course argue whether this is something useful, or even good practice, but we are not interested in that… yet) if it supports the ANSI C standard. So after building a consistent formal system we can create actual, inconsistency paradoxes not like “Can we have a compiler of language X written in X” (if you have such questions read this really nice article) but like … well I just haven’t thought about it thoroughly but I can paraphrase a paradox we learned in CEID (unfortunately in Greek) “Is the set {Turing machines which provably halt on every input} enumerable or non-enumerable?” just replace Turing Machine with compiler in this set

Ok boring… Just tell me which came first

Read more

Last week I needed to run some benches on a cpu. So besides the well known dhrystone cpu benchmark, I decided to also use coremark[wikipedia entry].

I won’t go into too many details about the coremark, since I want to focus this article into its poor web programming  practices. Coremark website probably isn’t the next facebook in terms of hits per second, nor the next youtube in terms of bandwidth utilization, yet it is a decent site that comes handy for many engineers willing to use their tool.

Bad practice 1 : The registration process

In order to download the coremark software, and its documentation you need to be a registered user. The registration process is a simple one, (probably less than 10-15 lines of php code) and another 100 lines of some opensource captcha php image generation.

Read more

As stated earlier, zentyal is a full fledged routing platform (a lot more actually), but that’s not my point. The point is that after setting it up some bits and pieces were out of place. From a quick review after installation, a newcomer will notice that there doesn’t exist some overview monitor about networking. Also there doesn’t exist a gui approach to setup a secure access-point, and not all dynamic dns providers are covered to use with zentyal. In this post I’ll describe all the necessary steps that I took to fix those problems.

Read more

Well since my previous post the internet had hiccups, the routing was segmented, the seven stages of hell was unleashed (since IDS, antivirus and actual filtering was dropped down) to my small home lan. it was high time to change pfsense distribution which had frustrated me in the past like no other. There were 2 alternatives: First install a bare gentoo system, configure it and let it fly. Most of the necessary (hostapd, ntpd, dnsmasq, squid, fail2ban) parts I already had experience on setting them up, but I didn’t had experience in other important parts like snort, and havp. Things could go messy, but nevertheless I moved to roll out this plan. While I was compiling for the router (a single core amd sempron 2800+ : Sempron 2800+ @ 2000 MHz with 256 KB L2, 333 MT/s FSB, 12x multiplier, 1.60 Vdd) using fully distcc and ccache to speed up everything, I started downloading the following distros to test them under a virtualbox instance. To be frank I knew , that, setting up a routing gentoo platform in old hardware would be an impeccable act of faith, to myself, the distribution but foremost the hardware itself, but the reward would be a top-notch networking performance given the hardware.

Read more

This is a quick blog post in order to set things straight. Though all major results from google pointing to the contrary (that linux is uninstallable on a sparc machine) these days I came to a fully functional 480 server while using gentoo sparc autobuilds. Other linux distros (debian,redhat,centos,suse) could not even boot the machine :S

So gentoo is your friend.

If for any reason anyone is intrested on the specifics of the “operation” please write a follow up


...
configure: Using system-installed FFMpeg code
configure: WARNING:
======================================================================
WARNING: you have chosen to build gst-ffmpeg against a random
external version of ffmpeg instead of building it against the tested
internal ffmpeg snapshot that is included with gst-ffmpeg.

This is a very bad idea. So bad in fact that words cannot express
just how bad it is. Suffice to say that it is BAD.

The GStreamer developers cannot and will not support a gst-ffmpeg
built this way. Any bug reports that indicate there is an external
version of ffmpeg involved will be closed immediately without further
investigation.

The reason such a setup can't be supported is that the ffmpeg API
and ABI is in constant flux, yet there aren't any official releases
of the ffmpeg library to develop against. This makes it impossible
to guarantee that gst-ffmpeg will work reliably, or even compile,
with a randomly picked version ffmpeg. Even if gst-ffmpeg compiles
and superficially appears to work fine against your chosen external
ffmpeg version, that might just not be the case on other systems, or
even the same system at a later time, or when using decoders,
encoders, demuxers or muxers that have not been tested.

Please do not create or distribute binary packages of gst-ffmpeg
that link against an external ffmpeg. Thank you!
======================================================================

checking for sed... /bin/sed
...

LOL that’s what I call programmers with humor And of course build succeeds and ffmpeg packages works alright

These days I am violating my shiny new pc in many ways (huge matlab datasets processing, with multiple concurrent vms running at 100%) so I would like to check out (without rebooting and entering the bios) some shit about my machine. Since I am totally F/OSS as far as the OS is concerned I don’t have the luxury of windows apps like cpuid, or whatever. So in order to check out about the o/c I had to find another way. And there is another way, two to be exact.

• The first is the lshw [1] progie (you know the drill gentoo fans ) emerge -vuDtN sys-apps/lshw

It is a good idea to build the frontend also (via the gtk USE flag). lshw is great. it provides you with tons of information, the cli’s ouput may passed as input (piped?) for anykind of shit you have in mind in anykind of enviroment, and is open source code. lshw (also known as Hardware Lister) is a small tool to provide detailed information on the hardware configuration of the machine. It can report exact memory configuration, firmware version, mainboard configuration, CPU version and speed, cache configuration, bus speed, etc. on DMI-capable x86 and amd64 (tested on both works flawlessly). For instance,

phenom2 ~ # lshw -class cpu
*-cpu
description: CPU
product: AMD Phenom(tm) II X4 940 Processor
vendor: Advanced Micro Devices [AMD]
physical id: 4
bus info: cpu@0
version: AMD Phenom(tm) II X4 940 Processor
serial: To Be Filled By O.E.M.
slot: CPU 1
size: 3400MHz
capacity: 3400MHz
width: 64 bits
clock: 200MHz
capabilities: fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp x86-64 3dnowext 3dnow constant_tsc rep_good nopl pni monitor cx16 popcnt lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt

• The second one is  a tool [2- 64bits]  [3 - 32bits] [4 - link to the amd’s search page] build by amd and for amd compatible cpus (haven’t tested it in intel archs though it would be funny) but unfortunatelly amd (what a surprise!) doesn’t provide exactly what one may want.

What I mean by that? If you download one of these distributions you  will find out that these are rpms [for those stuck with ebuilds and gentoo you can read more about rpm there 5] for two specific distros; Suse and Redhat! No gentoo, no slack no nothing… Not even a static build… Who the hell is the software release engineerer in amd? Of course there is a bypass to the rpm shit .

• One may extract the files out of the stupid rpms easily though;
  1. rpm2cpio CPUInfo-2_2_0_42-SUSE10364bit-Public.bin.rpm | cpio -idmv
  2. if you don’t have cpio (for gentoo users that’s impossible since its in the system) install it and proceed as advised
• and of course the extracted files are dynamically linked (YOU M#*$#@)DUMB*#$#*$)@# PLEASE PROVIDE US STATICALLY LINKED BINARIES OR GIVE OUT THE DAMNED SOURCE,s o it may or may not work in your platform depending on the libraries you have installed… sigh… (For me every single one worked)
  • CPUInfo is the "fantastic” executable the amd provides
phenom2 dumb-developers # ldd CPUInfo
linux-vdso.so.1 => (0×00007fffd35fe000)
…[many libraries including kdemultimedia!!!]…
libuuid.so.1 => /lib/libuuid.so.1 (0×00007f0dc65a9000)

Instead of a conclusion;  Linux users do yourself a favour and use truly opensource software. Not some binary shitty thing that a 13yr old kiddo written… And if you haven’t got the clue, USE lshw.

PS. If you dl’ed the amd’s rpm’s then you should see something similar to :

phenom2 amd # ls -l
σύνολο 6470
-rw-r--r-- 1 root root 611293 2008-11-18 10:52 CPUInfo-2_2_0_42-RedHatEnterpriseServer4U464bit-Public.bin.rpm
-rw-r--r-- 1 root root 1075290 2008-11-18 10:52 CPUInfo-2_2_0_42-RedHatEnterpriseServer564bit-Public.bin.rpm
-rw-r--r-- 1 root root 656880 2008-11-18 10:52 CPUInfo-2_2_0_42-SUSE102OSS64bit-Public.bin.rpm
-rw-r--r-- 1 root root 671294 2008-11-18 10:52 CPUInfo-2_2_0_42-SUSE10364bit-Public.bin.rpm


First of all, for those wondering, [1]: vectorization is a process where a sequential program using a pair of operands on the same instruction in an iteration is transformed into a vector program where a single instruction can perform multiple operations or a pair of vector. In this notation by vector I call datasets of the same type.

In quite modern machines/architectures (namely x86) there exists such a vector processing(one or more ) unit utilized by the SIMD instructions set (usually SSE,SSE, see below). SIMD [2] is one of various ways implementing parallelism in computer hardware.  In this schema,some large set of CPU components perform the same task (operation) at the same time, each with different data leading to a high utilization of these components without wasting CPU cycles in the same operation.

For instance the following code is vectorized:

#define SIZE 1024
int a[SIZE], b[SIZE], c[SIZE], d[SIZE];

void linear ()
{
int i;

for (i=0; i<SIZE; i++)
{
a[i] = b[i] * c[i] + d[i];
}
}
If the parallelism is viable, then the compiler after the autovectorization process may produce  high performance code that makes high utilization of processor vector units. This certaintly is of supreme importance (to reduce the total instructions executed where applicable) in a high performance distribution. So after some reasearch,  I decided to enable on all of my gentoo machines the vectorization on the CFLAGS (the vectorization as one may see in its homepage is automatically enabled on the -O3 optimizations [3] ) . Gentoo should be the fastest linux distro around; END OF STORY.

CFLAGS="-march=nocona -mtune=nocona -O2 -pipe -fomit-frame-pointer -ftree-vectorizer-verbose=5 -ftree-vectorize -fassociative-math -msse -msse2 -msse3"
The options to do so is the -ftree-vectorize combined with tha -msse -msse2 options (which are the SIMD utilizers for the x86 (amd64) arch) if you are having a more capable SIMD instruction set you may use it (i.e. -msse3, -msse4.1, -m3dnow(:-P) etc), while -fassociative-math can be used instead of -ffast-math to enable vectorization of reductions of floats.

here are some before and after results concerning vectorization speedup  on:

rodos ~ # uname -a
Linux rodos 2.6.27-gentoo-r8-korki-pentium-prescott #8 SMP Mon Mar 9 19:28:11 EET 2009 x86_64 Intel(R) Pentium(R) 4 CPU 3.00GHz GenuineIntel GNU/Linux

The results follow in the following snippet and are for 8192 bytes in kilo

Vectorize Enabled Vectorize Disabled
md2 7135,23 6435,79 0,11
mdc2 9210,54 9404,42 -0,02
md4 901018,97 636548,84 0,42
md5 546226,18 306845,91 0,78
hmac(md5) 551026,69 551318,87 0
sha1 299592,36 296080,73 0,01
rmd160 174093,65 148373,5 0,17
rc4 439552,68 439959,55 0
blowfish 99672,06 98432,34 0,01
aes-128 111878,14 111209,13 0,01
aes-192 102440,96 100368,38 0,02
aes-256 91635,71 91280,73 0


or in a more comprehensible representation:

Hi there all,

in this article we will cover how to  download and transcode into any format{avi,mp3,…} in a few commands This article is divided into two pieces; The init phase where all the necessary tools are build, and then some basic explanation of the tools usage.

We begin this article with the usual update your portage tree with
#emerge --sync
next you have to download the youtube-dl package. While it is possible to create a similar script with more or less functionality (i.e. using python and browser @ mechanize package) the point here is to be as simple as possible. So we download the package with
#emerge youtube-dl
Then we have to download some packages to manipulate the flv videos. Such tools (but not limited to) are ffmpeg and transcode. We download each one with the appropriate emerge command; Keep in mind that we have to setup the build flags on each one in order to have the ability to manipulate more than one datafile. In my system the /etc/portage/package.use contains the following on ffmpeg and transcode;
media-video/transcode X iconv jpeg mmx sse sse2 truetype 3dnow a52 altivec dv dvdread extrafilters fame imagemagick lzo mjpeg mp3 mpeg network ogg oss quicktime sdl theora\
v4l2 vorbis xml xvid

media-video/ffmpeg a52 aac amr encode ieee1394 imlib mmx network ogg oss sdl theora threads truetype v4l vorbis x264 xvid zlib
after setting up the /etc/portage/package.use for our tools then we emerge our packages with;

#emerge ffmpeg
#emerge transcode

This concludes our init/setup phase. And now we enter the tools usage part.
Scenario a.
Since we have forgot our Load CD and we desperately want to listen to until it sleeps track. No stress dude! This is a 1 minute task!

1. We locate the video in the youtube (eg http://www.youtube.com/watch?v=sZtszXnG11E ) then we call the youtube-dl with argument this url; $youtube-dl -b -t http://www.youtube.com/watch?v=sZtszXnG11E (-b is for the best quality -t is for the title, since we only wanted to make an mp3 from this track we could omit -b argument with no significant loss in the quality. -t is to create a filename which contains part of the video title). If we download with -b switch bear in mind that our media is saved as mp4 video file. at programs exit the youtube-dl reports the file that has been saved eg;
   
Retrieving video data: 100.0% ( 17.21M of 17.21M) at 125.20k/s ETA 00:00 done.
Video data saved to sZtszXnG11E.mp4

   the sZtszXnG11E.mp4 is the filename we are gonna tamper
2. We transcode the sZtszXnG11E.mp4 with ffmpeg with one simple command; $ffmpeg -i sZtszXnG11E.mp4 -f mp4 -vn -acodec copy UntilItSleeps.mp3
   1. If we had downloaded the same file without -b switch the input file (-i sZtszXnG11E.mp4  in the previous bullet) would be something like sZtszXnG11E.flv. In that case we change the input codec parameter from mp4 to mp3 ; $ffmpeg -i sZtszXnG11E.flv -f mp3 -vn -acodec copy UntilItSleeps.mp3. We do so because the audio data are represented differently in flv (mp3) and mp4 (mp4) files respectively.

If anyone has any other scenario in mind please ask