Wednesday, March 3, 2010

More on Swapping Vs. Paging

Just using a lot of swap space doesn’t necessarily mean that you need more memory. Here’s how to tell when Linux is happy with the available memory and when it needs more.

Linux novices often find virtual memory mysterious, but with a grasp of the fundamental concepts, it’s easy to understand. With this knowledge, you can monitor your system’s memory utilization using vmstat and detect problems that can adversely affect system performance.
How Virtual Memory Works

Physical memory-the actual RAM installed-is a finite resource on any system. The Linux memory handler manages the allocation of that limited resource by freeing portions of physical memory when possible.

All processes use memory, of course, but each process doesn’t need all its allocated memory all the time. Taking advantage of this fact, the kernel frees up physical memory by writing some or all of a process’ memory to disk until it’s needed again.

The kernel uses paging and swapping to perform this memory management. Paging refers to writing portions, termed pages, of a process’ memory to disk. Swapping, strictly speaking, refers to writing the entire process, not just part, to disk. In Linux, true swapping is exceedingly rare, but the terms paging and swapping often are used interchangeably.

When pages are written to disk, the event is called a page-out, and when pages are returned to physical memory, the event is called a page-in. A page fault occurs when the kernel needs a page, finds it doesn’t exist in physical memory because it has been paged-out, and re-reads it in from disk.

Page-ins are common, normal and are not a cause for concern. For example, when an application first starts up, its executable image and data are paged-in. This is normal behavior.

Page-outs, however, can be a sign of trouble. When the kernel detects that memory is running low, it attempts to free up memory by paging out. Though this may happen briefly from time to time, if page-outs are plentiful and constant, the kernel can reach a point where it’s actually spending more time managing paging activity than running the applications, and system performance suffers. This woeful state is referred to as thrashing.

Using swap space is not inherently bad. Rather, it’s intense paging activity that’s problematic. For instance, if your most-memory-intensive application is idle, it’s fine for portions of it to be set aside when another large job is active. Memory pages belonging to an idle application are better set aside so the kernel can use physical memory for disk buffering.
Using vmstat

vmstat, as its name suggests, reports virtual memory statistics. It shows how much virtual memory there is, how much is free and paging activity. Most important, you can observe page-ins and page-outs as they happen. This is extremely useful.

To monitor the virtual memory activity on your system, it’s best to use vmstat with a delay. A delay is the number of seconds between updates. If you don’t supply a delay, vmstat reports the averages since the last boot and quit. Five seconds is the recommended delay interval.

To run vmstat with a five-second delay, type:

vmstat 5

You also can specify a count, which indicates how many updates you want to see before vmstat quits. If you don’t specify a count, the count defaults to infinity, but you can stop output with Ctrl-C.

To run vmstat with ten updates, five seconds apart, type:

vmstat 5 10

Here’s an example of a system free of paging activity:

procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
0 0 0 29232 116972 4524 244900 0 0 0 0 0 0 0 0 0
0 0 0 29232 116972 4524 244900 0 0 0 0 2560 6 0 1 99
0 0 0 29232 116972 4524 244900 0 0 0 0 2574 10 0 2 98

All fields are explained in the vmstat man page, but the most important columns for this article are free, si and so. The free column shows the amount of free memory, si shows page-ins and so shows page-outs. In this example, the so column is zero consistently, indicating there are no page-outs.

The abbreviations so and si are used instead of the more accurate po and pi for historical reasons.

Here’s an example of a system with paging activity:

procs memory swap io system cpu
r b w swpd free buff cache si so bi bo in cs us sy id
. . .
1 0 0 13344 1444 1308 19692 0 168 129 42 1505 713 20 11 69
1 0 0 13856 1640 1308 18524 64 516 379 129 4341 646 24 34 42
3 0 0 13856 1084 1308 18316 56 64 14 0 320 1022 84 9 8

Notice the nonzero so values indicating there is not enough physical memory and the kernel is paging out. You can use top and ps to identify the processes that are using the most memory.

You also can use top to show memory and swap statistics. Here is an example of the uppermost portion of a typical top report:

14:23:19 up 348 days, 3:02, 1 user, load average: 0.00, 0.00, 0.00
55 processes: 54 sleeping, 1 running, 0 zombie, 0 stopped
CPU states: 0.0% user, 2.4% system, 0.0% nice, 97.6% idle
Mem: 481076K total, 367508K used, 113568K free, 4712K buffers
Swap: 1004052K total, 29852K used, 974200K free, 244396K cached

For more information about top, see the top man page.

It isn’t necessarily bad for your system to be using some of its swap space. But if you discover your system is often running low on physical memory and paging is causing performance to suffer, add more memory. If you can’t add more memory, run memory-intensive jobs at different times of the day, avoid running nonessential jobs when memory demand is high or distribute jobs across multiple systems if possible.

Tuesday, February 23, 2010

20 Linux System Monitoring Tools Every SysAdmin Should Know

Solaris Swapping and Paging

Displaying %ENV completely

#! /usr/bin/perl -w

use strict;

my $keyy;

foreach $keyy (keys %ENV)

print "$keyy and its value = $ENV{$keyy} \n";



Perl use lib, unshift difference

What's the difference between use, unshift and PERL5LIB when adding/including module path?


use lib - is a compile time inclusion. Try this example:

#! /usr/bin/perl -w

use strict;

print "@INC\n";
use lib '/home/kdranjan/Perl/Dynamic';
Print "@INC\n";

Run it and it will display: (Both are same because added path was included at compile time)
- /home/kdranjan/Perl/Dynamic /etc/perl /usr/local/lib/perl/5.10.0 /usr/local/share/perl/5.10.0 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl .

- /home/kdranjan/Perl/Dynamic /etc/perl /usr/local/lib/perl/5.10.0 /usr/local/share/perl/5.10.0 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl .


unshift - is a run time inclusion.

Modify previous script a bit like this:
#! /usr/bin/perl -w

use strict;

print "@INC\n";
unshift(@INC, '/home/kdranjan/Perl/Dynamic');
print "@INC\n";

- /etc/perl /usr/local/lib/perl/5.10.0 /usr/local/share/perl/5.10.0 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl .

- /home/kdranjan/Perl/Dynamic /etc/perl /usr/local/lib/perl/5.10.0 /usr/local/share/perl/5.10.0 /usr/lib/perl5 /usr/share/perl5 /usr/lib/perl/5.10 /usr/share/perl/5.10 /usr/local/lib/site_perl .

Sunday, February 21, 2010

How does gcc, ld and g++ work

Thursday, February 18, 2010

Defunct (Zombie) process, preap and orphan process

A defunct (or zombie) process is one whose exit status has yet to be reaped by its parent. So when a process shows as "defunct" in ps, you need to reap it. Here's how:

preap(1) - force a defunct process to be reaped by its parent

Syntax: /usr/bin/preap PID

So, to get rid of all zombies on our system, all we have to do is script this to reap all process marked as defunct:

/usr/bin/preap $(ps -ef | grep defunct | awk '{ print $2 }' | xargs)

So, what's an orphan process then? If the parent of a defunct process happens to exit, it becomes an orphan, and is asigned PID 1 as a parent - see init(1M).

Solaris: How to know all the options of a command from process id

Question: How to find out, What all options or inputs were specified with a unix tool, from process id?

Ans: Solaris supports 'pargs', which, when, run with , will throw all the inputs specified with the command. Question is, why is 'ps -eaf' not sufficient. Answer lies in what 'ps -eaf' prints. 'ps -eaf' can print only limited number of outputs on one line, mostly, 80 char. What if, command + inputs were bigger/wider than 80 chars.

Also, 'pargs -e can also print the environment variable path.

Wednesday, February 10, 2010

mod_bandwidth in Apache Web Servers

Q. How can we deal with the situation where we can afford only a limited amount of bandwidth but some of the service's content is large such as streaming media or large files?

A. mod_bandwidth is a Apache module that enables the setting of server-side wide or per-connection bandwidth limits, based on directory, size of files, and remote IP/Domain.

Apache Load Balancing

Load balancing can help distribute load off of one Apache Web Server machine to logically and geographically distributed Apache Web Servers.

Load balancing can be achieved using 'mod_backhand', Cisco LocalDirector.

URL for mod_backhand:

User who are really serious about tuning their Web Servers can refer to
'Web Performance Tuning' by Patrick Killelea (O'Reilly).

Tuesday, February 9, 2010

What is Taint mode in perl

Single Line Swap in Perl

($foo, $bar) = ($bar, $foo)

will work just fine.

Perl 5.6 Vs 5.8

Perl 5.6.1 and Perl 5.8.x are not binary compatible (It means that user will have to download the source code in c and install it using their c development and build environment). Binary versions may be available directly from vendors such as RedHat or Ubuntu etc.

There are a few important differences to consider:

- Smaller installation,
- Faster in some cases,
- Broken threading model,
- No unicode

- Larger installation,
- Threading model mostly works,
- Has unicode support

Highlights In 5.8.0

- Better Unicode Support:
Unicode support has been much enhanced since 5.6, at all levels:
- now supports Unicode 3.2.0 (5.6.1 supports 3.0.1)
- at the language (and internals) level Unicode support is
now more ubiquitous and robust
- regular expressions now work with Unicode
- support for non-Latin encodings (such as the various
Chinese/Japanese/Korean encodings) through the Encode module

- New Threads Implementation:
A new multithreading implementation called interpreter threads,
or "ithreads" for short, is available, their use instead of the
old "5.005 threads" is strongly encouraged. The major difference
is that in ithreads any data sharing must be done explicitly.

- New IO Implementation:
the new PerlIO implementation is both a portable stdio implementation
(at the source code level) and a flexible new framework for richer
I/O behaviours

- Better Numeric Accuracy:
previous Perls relied on vendors' string-to-number and back
routines which in some cases proved to be too much trust
leading to nonportable and wrong behaviours

- 64-bit support:
64-bit support is now considered to be mature -- if your platform
supports 64-bit integers or address space, you can compile Perl to
use those

- Safe Signals:
in previous versions of Perl signals could corrupt Perl's internal state

- Many New Modules:
Digest::MD5, File::Temp, Filter::Simple, libnet, List::Util,
Memoize, MIME::Base64, Scalar::Util, Storable, Switch,
Test::More, Test::Simple, Text::Balanced, Tie::File, ...

- Extensive Regression Testing:
Perl has now almost six times as many tests as in 5.6,
and the code is test built daily on several platforms


mainly because of the PerlIO introduction, Perl 5.8 is not
binary compatible with any earlier Perl release, XS MODULES

- AIX Dynaloading:
Perl uses now AIX dynaloading, instead of the older emulated
version, to be more compatible with other applications on AIX

- 64-bit Platforms No Longer Use Perl Malloc:
the Perl malloc seems to have various problems on platforms
with 64-bit addressing, therefore the default in these cases
is to use the native malloc

- Hashing Order Changed Once Again:
the function used in the implementation of hashes was changed
to a better one once again, but your code shouldn't be expecting
any particular key ordering

- Attributes For my Now Handled At Run-Time:
the attributes for my() are now run-time, as opposed to compile time

- REF(...) instead of SCALAR(...):
to be consistent with ref()'s results, references to references
now stringify as "REF(...)"

- Unicode Model Changed (no more "use utf8", almost)
In Perl 5.6 "Unicodeness" was lexically scoped to the operations;
in Perl 5.8 "Unicodeness" is bound to the data. The only remaining
use of "use utf8" is when the Perl script itself is written in the
UTF-8 encoding of Unicode.

- VMS: Socket Extension Dynamic, IEEE fp Default on Alpha
- the Socket extension is now dynamic rather than static, which may
cause problems in really old VMS installations
- the IEEE floating point is now the default format in OpenVMS Alpha,
see README.vms for reasons and other details

Nomenclature Change

- What the "Camel III" book called an "IO discipline"
is now called an "IO layer"


- dump():
the functionality of the dump command is now considered obsolete

- 5.005 threads are now to be considered deprecated;
the new "interpreter threads" implementation should be used instead

- Pseudohashes:
the user-visible implementation of pseudohashes is going to be removed
and replaced with something cleaner (also, the internal implementation
will have to go since it was found to slow down the overall hash access)

- Use of tainted data in exec LIST and system LIST:
now gives a warning, but will become fatal error in a future release

- tr///C, tr///U:
the interface was found to be a mistake, pack("C0", ...) and
pack("U0", ...) can be used instead

Wednesday, February 3, 2010

Crontab - Good Reference

Crontab – Quick Reference

Setting up cron jobs in Unix and Solaris

cron is a unix, solaris utility that allows tasks to be automatically run in the background at regular intervals by the cron daemon. These tasks are often termed as cron jobs in unix , solaris. Crontab (CRON TABle) is a file which contains the schedule of cron entries to be run and at specified times.

This document covers following aspects of Unix cron jobs
1. Crontab Restrictions
2. Crontab Commands
3. Crontab file – syntax
4. Crontab Example
5. Crontab Environment
6. Disable Email
7. Generate log file for crontab activity

1. Crontab Restrictions
You can execute crontab if your name appears in the file /usr/lib/cron/cron.allow. If that file does not exist, you can use
crontab if your name does not appear in the file /usr/lib/cron/cron.deny.
If only cron.deny exists and is empty, all users can use crontab. If neither file exists, only the root user can use crontab. The allow/deny files consist of one user name per line.

2. Crontab Commands

export EDITOR=vi ;to specify a editor to open crontab file.

crontab -e Edit your crontab file, or create one if it doesn’t already exist.
crontab -l Display your crontab file.
crontab -r Remove your crontab file.
crontab -v Display the last time you edited your crontab file. (This option is only available on a few systems.)

3. Crontab file
Crontab syntax :
A crontab file has five fields for specifying day , date and time followed by the command to be run at that interval.

* * * * * command to be executed
- - - - -
| | | | |
| | | | +----- day of week (0 - 6) (Sunday=0)
| | | +------- month (1 - 12)
| | +--------- day of month (1 - 31)
| +----------- hour (0 - 23)
+------------- min (0 - 59)
* in the value field above means all legal values as in braces for that column.
The value column can have a * or a list of elements separated by commas. An element is either a number in the ranges shown above or two numbers in the range separated by a hyphen (meaning an inclusive range).
A. ) Repeat pattern like /2 for every 2 minutes or /10 for every 10 minutes is not supported by all operating systems. If you try to use it and crontab complains it is probably not supported.

B.) The specification of days can be made in two fields: month day and weekday. If both are specified in an entry, they are cumulative meaning both of the entries will get executed .

4. Crontab Example
A line in crontab file like below removes the tmp files from /home/someuser/tmp each day at 6:30 PM.

30 18 * * * rm /home/someuser/tmp/*

Changing the parameter values as below will cause this command to run at different time schedule below :

min hour day/month month day/week Execution time
30 0 1 1,6,12 * – 00:30 Hrs on 1st of Jan, June & Dec.
0 20 * 10 1-5 –8.00 PM every weekday (Mon-Fri) only in Oct.
0 0 1,10,15 * * – midnight on 1st ,10th & 15th of month
5,10 0 10 * 1 – At 12.05,12.10 every Monday & on 10th of every month
Note : If you inadvertently enter the crontab command with no argument(s), do not attempt to get out with Control-d. This removes all entries in your crontab file. Instead, exit with Control-c.

5. Crontab Environment
cron invokes the command from the user’s HOME directory with the shell, (/usr/bin/sh).
cron supplies a default environment for every shell, defining:

Users who desire to have their .profile executed must explicitly do so in the crontab entry or in a script called by the entry.

6. Disable Email
By default cron jobs sends a email to the user account executing the cronjob. If this is not needed put the following command At the end of the cron job line .

>/dev/null 2>&1

7. Generate log file
To collect the cron execution execution log in a file :

30 18 * * * rm /home/someuser/tmp/* > /home/someuser/cronlogs/clean_tmp_dir.log


Crontab – Quick Reference

Posted using ShareThis

Thursday, January 28, 2010

Daemon a perl process

Another method: