Linux Memory Usage

I recently became concerned about the performance of a particular application (cmatrix on the 1080p screen – it looks beautiful but it’s quite choppy) so I began some investigating and turned up some surprising statistics.

Uptime: 12:53; of that time, Chrome has been running for most of it and we all know web browsers are notorious for leaking all over your RAM.  Chrome is better than firefox but still pretty leaky.  On Windows XP, it was not uncommon for chrome to be using more than half a GB of RAM.

Now, the surprising thing is, Linux has managed to gobble up all my RAM.  But has it?  If you observe carefully, I’ll explain to you why Linux has -not- gobbled up all of my RAM (despite what ‘top’ tells me) whereas Windows, had it been given the same constraints, would’ve.

The way Linux handles memory usage differs extraordinarily to Windows.  If something leaks in Windows, that RAM is unrecoverable, especially after the application has quit.  However, in Linux, if something leaks, once the application quits, all the memory Linux allocated it is returned to Linux in the form of ‘cached’ memory.  In other words, the memory doesn’t get -deleted-, so if the application comes back and wants it again it’s entitled to the same memory (efficiency++) but if another application wants to use the memory that application -was- using, Linux isn’t gonna say no.  In other words, the memory is still ‘in use’ but it’s also still ‘available’ to new applications who request it.

This presents a difficult situation when you’re measuring RAM usage – do you count the memory as ‘in use’ or as ‘available’?  Well, ‘top’ (and some other tools) see it as ‘in use’, and ‘free’ will also tell you (on the Memory line) that it’s ‘in use’.  However, ‘free’ also has a “-/+ buffers/cache” line which basically takes the buffered memory into account, giving you a fuller picture of how much RAM is available to new applications.  So when you’re measuring RAM, you should use -that- line in your statistics.  Not the above raw figure which counts unreferenced (‘dead’) memory as ‘used’.

Windows XP, on the other hand (but don’t quote me about Vista or 7, I’m unfamiliar with the new kernel but I’m pretty sure it’s the same story in terms of memory allocation), trusts applications to free memory.  If they don’t, Windows will never reclaim the lost memory (not until a reboot anyway).  I’m unsure exactly how the kernel works (only Microsoft know) for this behaviour to be apparent, but I’m not the only one to have made that observation, and I can assure you Windows XP, if left to its own devices, will start swapping out pages like there’s no tomorrow after a couple of days (depending how much RAM you have).

Suffice it to say, if you run WinXP, you’re gonna get slowdown after a while, if you run Linux you can (in theory) continue running it forever.  Note that kernel leaks (which are rare but do happen) and certain other situations (eg video driver bugs) can cause a required reboot due to memory loss, but the situation is far rarer in the Linux world than in Windows.

It is owing mainly to this reason (and the kernel’s greater stability, owing to a better design and fewer crash-inducing bugs) that Linux servers are able to run for…

chris@w1zard:~$ uptime
20:54:58 up 125 days,  4:24,  2 users,  load average: 0.00, 0.00, 0.00

Yup, w1zard (my server) has now been running for 125 days, 4 hours and 24 minutes without a reboot.  The load average statistics being 0.00 is due to the quad core processor; the system doesn’t strain under the minute amount of traffic it’s required to handle (mail only at the moment, too).  I’m very proud.
Have fun hacking!

Servers! Roof!

Yes, my servers are finally in the roof; which means I can relax and wipe everything off my whiteboard.  I’m happy.  I’m also connected through w1zard who is now our network’s personal Proxy server (currently optional, in the long run mandatory).  It caches and filters and makes life more streamlined and safer for the old fogeys in the house who are a little naive to say the least (my mother searched Google for “fisting”, thinking it was the act of shaking one’s fist; which first prompted me to think about setting up a filtering proxy).

Ossum has changed its name to Enosis and still hasn’t been worked on very much; but the infrastructure is now in place so the next step is to write a plan and then a tech demo.  Fun, fun, fun!

I’ve received four 7″x5″ canvases and finally dug out mum’s old acrylics (of which there is a disappointingly small pallet, but it should be significant for the work I have planned); I will be painting four abstract works entitled “Battlefield of the Mind”.  Pictures once I’ve actually done it.

I have also begun thinking about considering contemplating starting to ponder the possibility of organising a “Geeknic” over the summer, so if you’re a geek in the Tewkesbury area (or a geek who can reasonably travel to the Tewkesbury area) then get in touch, the more the merrier!

Very excited about finally having my desktop back…  And my whiteboard, for that matter!

Oh, and started work on a computerised Rubik’s Cube solver with the ultimate goal of making a robot that will solve rubik’s cubes (hopefully to be shown off at the geeknic).

Servers!

The two servers I ordered from ebay have arrived!  £235 for:

w1zard:

  • Intel Xeon 3.2GHz (/proc/cpuinfo reports 2 cores)
  • 1024MB of RAM (free reports 1023012 bytes)
  • 73.1GB 15k RPM SCSI hard disk drive (df reports 67456272 bytes on /)
  • 2x1Gb ethernet (both identical Intel 82541GI Gigabit Ethernet Controllers)
  • Floppy disk drive, CD ROM drive, front-availability VGA and 2xUSB

w1tch:

Same as above

W1zard is already up and running, serving you this webpage!  W1tch will be up when I work out how to configure the network (I’m thinking internet->router->w1zard->w1tch->war1ock for the time being, with w1tch and war1ock later becoming siblings in a w1zard-controlled network).

Problems encountered so far:

  1. Missing kettle leads for both servers (borrowed a couple, ordered some from Ebay so I can return the borroed cables)
  2. W1zard’s RAID was set to RAID 0, but the second disk had been removed – so I got a RAID error on first boot – solved by switching RAID off in the BIOS.
  3. Minor damage to w1tch’s front mount thingy, fixed by bending it back the couple of mil it was out by.

They are also very noisy, and very big; which doesn’t matter in the long run as their ultimate location will be the attic, but for now they are on my desk and very much in the way.

Both servers also came without any software, but the MBs support PXE; but it was a quicker job to use a Ubuntu 9.10 server edition disk on w1zard – I may experiment with PXE when I get w1tch up and running.

Any questions?