bad web programming practices: the coremark website

Last week I needed to run some benches on a cpu. So besides the well known dhrystone cpu benchmark, I decided to also use coremark[wikipedia entry].

I won’t go into too many details about the coremark, since I want to focus this article into its poor web programming  practices. Coremark website probably isn’t the next facebook in terms of hits per second, nor the next youtube in terms of bandwidth utilization, yet it is a decent site that comes handy for many engineers willing to use their tool.

Bad practice 1 : The registration process

In order to download the coremark software, and its documentation you need to be a registered user. The registration process is a simple one, (probably less than 10-15 lines of php code) and another 100 lines of some opensource captcha php image generation.

Read more

zentyal microconfigurations - setting up a secure AP, PPPoE, ntop, and dynamic dns

As stated earlier, zentyal is a full fledged routing platform (a lot more actually), but that’s not my point. The point is that after setting it up some bits and pieces were out of place. From a quick review after installation, a newcomer will notice that there doesn’t exist some overview monitor about networking. Also there doesn’t exist a gui approach to setup a secure access-point, and not all dynamic dns providers are covered to use with zentyal. In this post I’ll describe all the necessary steps that I took to fix those problems.

Read more

Checking out status quo in routing distributions

Well since my previous post the internet had hiccups, the routing was segmented, the seven stages of hell was unleashed (since IDS, antivirus and actual filtering was dropped down) to my small home lan. it was high time to change pfsense distribution which had frustrated me in the past like no other. There were 2 alternatives: First install a bare gentoo system, configure it and let it fly. Most of the necessary (hostapd, ntpd, dnsmasq, squid, fail2ban) parts I already had experience on setting them up, but I didn’t had experience in other important parts like snort, and havp. Things could go messy, but nevertheless I moved to roll out this plan. While I was compiling for the router (a single core amd sempron 2800+ : Sempron 2800+ @ 2000 MHz with 256 KB L2, 333 MT/s FSB, 12x multiplier, 1.60 Vdd) using fully distcc and ccache to speed up everything, I started downloading the following distros to test them under a virtualbox instance. To be frank I knew , that, setting up a routing gentoo platform in old hardware would be an impeccable act of faith, to myself, the distribution but foremost the hardware itself, but the reward would be a top-notch networking performance given the hardware.

Read more

random reboots - pfsense gone

It’s been a while since I’ve started using pfsense (almost 2 and a half years) and prior to this last month it operated flawlessly. This month though, it started having some hiccups, which led to random reboots (R). These days I have some spare time, and I feel uber productive (wrote 2 patches for pdfcrack and other minor work) I thought that this would be a nice opportunity to contribute some code to pfsense since I wanted to give something back to the tool.

I had ruled out any problems in hardware; I tested memory for 12 hours (memtest86 - during this period I had no internet :S) and another 6hours to check my disk using badblocks. I also wrote a something like a “watch” script that was calling mbm and threw most of its output to syslog (via logger). Then I had configured syslog to deliver logs to a remote host in order to overview it. Unfortunately nothing critical came out of the syslog, so I guessed that had something to do with the other components (kernel/packages)

I started looking the code and at the same time I logged in at their irc channel at irc (freenode / #pfsense) I had some minor issues with the tools especially some scripts that in my opinion caused the problem (1 change-set). Also some other minor issues were fixed like the relative reference of index.php at fbegin.inc etc. In total I had 4 change sets and a brand new recompiled FreeBSD kernel, which stabilized my system and made router not  randomly rebooting. At the same time I wrote to the irc about various stuff, when I understood that some pfsense packages couldn’t be uninstalled via the web-gui. That would be normal, since I was already getting my hands dirty, I got some guidelines from the pfsense irc channel (many thanks to operator jim-p-work) and then I tried to solve it my way (btw one may use the pfsense dev shell and there he can write a strange mixture of php and shell commands and execute them using an exec statement . Ugly I guess but probably useful.)

When I tried to use the shell things got messier. I was keeping an eye at :

  1. authgui.inc
  2. guiconfig.inc
  3. pkg-utils.inc
  4. pkg_mgr_install.php

and trying to find a solution to my problem. the idea was simple run similar things like the webgui but at the same time having an overview of the process. As I kept looking at the pfsense code of the above files a simple thing was coming to mind CSRF/XSS. People at the channel told me that it was the second time one reported such problems, and obviously enough everyone that has even worked for a month as a web-developer knows and can easily identify such a problematic code. To tell a long story short this happens by running server side scripts using variables passing via GET requests. The simplest scenario that comes to mind is that a misbehaving user can lure the admin to firstly open a pfsense webgui tab and then ask him to connect to one of his pages. Then by knowing simply the ip address of the router that the admin is connecting (on the private side ie 192.168.1.1)  is sufficient to mount the attack. In my review I looked only at pkg_mgr_install.php which a remote user could easily enough by having the above knowledge to uninstall snort and other mission critical packages. Currently there are at least 620 GET variable references in the pfsense and I am not quite sure on the security impact. I haven’t reviewed the whole codebase, and to be frank I am not willing to.

Why? The code is simply a mess. I am not sure if this is the effect of working with paranoid perfectionists in the past or if the code is simply ugly and unreadable. I wondered many times how someone can maintain such a codebase. I shared my concerns with GeekGod (aka sullrich @ pfsense) but the conversation was private and I intend to keep it that way unless he doesn’t mind sharing. After the small code review (less than 4hrs spent) in my opinion pfsense currently is “an accident waiting to happen” especially if you have some kind of open infrastructure.

Currently I am searching for something new to kick out pfsense, and for the first time, in my life besides the feature sets, I also review the code, to see if the project can be entrusted. So far I think ClearOS is better, with a much cleaner code base, but I will get back on that sooner than later with a small review on the webgui routing distros I’ve tried (I hope :P)

After all this is what open-source is all about, right?

Kernel choking, process killed. A flashback from 90s :)

[188144.606276] lowmem_reserve[]: 0 3255 8053 8053
[188144.606288] DMA32 free:24252kB min:4636kB low:5792kB high:6952kB active_anon:2755804kB inactive_anon:350564kB active_file:736kB inactive_file:380kB unevictable:0kB present:3334048kB pages_scanned:288 all_unreclaimable? no
[188144.606295] lowmem_reserve[]: 0 0 4797 4797
[188144.606306] Normal free:7368kB min:6832kB low:8540kB high:10248kB active_anon:4094700kB inactive_anon:682632kB active_file:1520kB inactive_file:1148kB unevictable:0kB present:4912640kB pages_scanned:1120 all_unreclaimable? no
[188144.606313] lowmem_reserve[]: 0 0 0 0
[188144.606319] DMA: 4*4kB 2*8kB 2*16kB 2*32kB 2*64kB 2*128kB 0*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15872kB
[188144.606336] DMA32: 4042*4kB 2*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 2*4096kB = 24376kB
[188144.606352] Normal: 781*4kB 3*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 7244kB
[188144.606368] 1337 total pagecache pages
[188144.606371] 0 pages in swap cache
[188144.606375] Swap cache stats: add 0, delete 0, find 0/0
[188144.606378] Free swap = 0kB
[188144.606381] Total swap = 0kB
[188144.658873] 2097136 pages RAM
[188144.658878] 59457 pages reserved
[188144.658881] 5356 pages shared
[188144.658883] 2021527 pages non-shared
[188144.658889] Out of memory: kill process 4574 (bash) score 40435 or a child
[188144.658929] Killed process 5193 (ipython)

Its ages since I have seen something similar. And imagine that this machine has 8Gbyte of RAM and 16Gbyte of swap :O

Update

Now I noticed that the swap was offline…. poor me :S

It’s been a while…

It nearly two months since I last wrote anything in my space (this one i mean :P), so I guess it’s time to update the few fellas that follow me electronically.

Not long ago, i was introduced into indifex, a company specializing into scalable content distribution technologies. I joined that fellowship and currently I am intrigued by its merits. First of all I got acquainted with django [1] which is a web design/implementation framework written in python. I really like python so this is, in principle, cool!

In the past few weeks every day is a new challenge workingwith something unique and ultra nice. For instance in the previous week I got involved with rabbitmq[2] and celery[3]. Rabbitmq is a distributed queueing system implementing AMQP protocol [4] (other cool servers are zeromq[5] - a really good comparison based on hard facts among them can be found here [6]) Celery on the other hand is a distributed task queue, designed -at first- for django projects. It is used for executing tasks asynchronously, routed to one or more worker servers on the same or distinct machine(s), running concurrently using multiprocessing on each one of them.

Another challenging task I faced was the overall design of the web services of transifex.net, using bleeding edge performance targeting applications like haproxy[7], nginx[8]. On this design many questions and objections were raised against the proposed design, but after all the architectural superiority prevailed and stayed.

Soon I will update with  the cloud (amazon ec2 and rackspace cloud) experience :)

sunfire 480R and linux

This is a quick blog post in order to set things straight. Though all major results from google pointing to the contrary (that linux is uninstallable on a sparc machine) these days I came to a fully functional 480 server while using gentoo sparc autobuilds. Other linux distros (debian,redhat,centos,suse) could not even boot the machine :S

So gentoo is your friend.

If for any reason anyone is intrested on the specifics of the “operation” please write a follow up :D

kOlga back online

as promised and within the 16hours deadline the kOlga is back online. The new feature set installation (hotspot, gardenwall, red queueing on all interfaces, etc) is postoponed for the future.

Enjoy the beautiful saturday and the weekend :mrgreen:

PS1. The tzikis link was decommissioned due to poor performance (4mbps up 4mbps down) but may be brought back if noone else provides a link for him.

PS2. To the guy that broadcasts in southwest patra region with mac address : 00:0B:6B:09:F2:7E and hidden SSID @ 5180Mhz please contact me if you interested for a link. My scans indicate a very good signal from your side (-55dbi with SNR of 50db)

The funniest configure message I’ve ever read :)


...
configure: Using system-installed FFMpeg code
configure: WARNING:
======================================================================
WARNING: you have chosen to build gst-ffmpeg against a random
external version of ffmpeg instead of building it against the tested
internal ffmpeg snapshot that is included with gst-ffmpeg.


This is a very bad idea. So bad in fact that words cannot express
just how bad it is. Suffice to say that it is BAD.


The GStreamer developers cannot and will not support a gst-ffmpeg
built this way. Any bug reports that indicate there is an external
version of ffmpeg involved will be closed immediately without further
investigation.

The reason such a setup can't be supported is that the ffmpeg API
and ABI is in constant flux, yet there aren't any official releases
of the ffmpeg library to develop against. This makes it impossible
to guarantee that gst-ffmpeg will work reliably, or even compile,
with a randomly picked version ffmpeg. Even if gst-ffmpeg compiles
and superficially appears to work fine against your chosen external
ffmpeg version, that might just not be the case on other systems, or
even the same system at a later time, or when using decoders,
encoders, demuxers or muxers that have not been tested.

Please do not create or distribute binary packages of gst-ffmpeg
that link against an external ffmpeg. Thank you!
======================================================================


checking for sed... /bin/sed
...

LOL that’s what I call programmers with humor :twisted: And of course build succeeds and ffmpeg packages works alright :D

kOlga administratively down

the wifi node due to maintenance is down and will be down for at least 16hours. 

There will be installed new ap cards, new casing and some new cool features. 

 My apologies for the inconvenience

 

Cheers