Wednesday, May 25, 2011

tar

The compression option provides a more efficient way of expressing:
tar -cvf - directory | mkszip >archive
as the one command line:
tar -cvzf archive directory
To identify all files that have been changed in the last week (7 days), and to archive them to a file on diskette, you might type:
find directory -mtime -7 | tar -cvf a:archive -

gzip

Compressing Multiple Files

You can concatenate multiple compressed files. When you do so, gunzip (or gzip -d) extracts all files in the compressed file as a single file. For example:
gzip -c file1  > catfiles.gz
gzip -c file2 >> catfiles.gz
After creating the compressed file catfiles.gz, the command:
gunzip -c catfiles.gz
is equivalent to
cat file1 file2
If one of the files in such a .gz file is damaged or corrupt, the other files can still be recovered (if the damaged or corrupt member is removed).
You can improve the level of compression achieved by compressing all the files at once rather than compressing them individually and then concatenating the results. For example:
cat file1 file2 | gzip > catfiles.gz
yields better compression than:
gzip -c file1 file2 > catfiles.gz
You can recompress concatenated files to get better compression with a command like:
gzip -cd old.gz | gzip > new.gz
When a compressed file contains several individual files, the uncompressed size and CRC reported by the --list option are for the last member only. To get the uncompressed size for all members, use:
gzip -cd file.gz | wc -c
Multiple compressed files can be concatenated. In this case, gunzip will extract all members at once. For example:

gzip -c file1 > foo.gz
gzip -c file2 >> foo.gz
Then

gunzip -c foo
is equivalent to

cat file1 file2
In case of damage to one member of a .gz file, other members can still be recovered (if the damaged member is removed). However, you can get better compression by compressing all members at once:

cat file1 file2 | gzip > foo.gz
compresses better than

gzip -c file1 file2 > foo.gz
If you want to recompress concatenated files to get better compression, do:

gzip -cd old.gz | gzip > new.gz
If a compressed file consists of several members, the uncompressed size and CRC reported by the --list option applies to the last member only. If you need the uncompressed size for all members, you can use:

gzip -cd file.gz | wc -c
If you wish to create a single archive file with multiple members so that members can later be extracted independently, use an archiver such as tar or zip. GNU tar supports the -z option to invoke gzip transparently. gzip is designed as a complement to tar, not as a 
One other useful option is the-rflag, which tellsgzipandgunzipto recursively compress or decompress all files in the current directory and any subdirectories. (Even with the-rflag,gzipstill compresses one file at a time.) Here are some examples:
gzip -r somedirZip all files in thesomedirdirectory.
gunzip -r somedirUnzip all files in thesomedirdirectory.

Handling Compressed Archives
It's common to applygzipto a tar file, which is why you see files with names likesomething.tar.gzon Linux systems. When you want to extract the contents of a gzipped tar file, you have several choices. The first is to usegunzipfollowed bytar, like this:
gunzip something.tar.gz
tar xvf something.tar

Or you could do it all in one command, like this:
gunzip -c something.tar.gz | tar xvf -
The-cflag tellsgunzipto decompress the file, but instead of creating asomething.tarfile, it pipes the decompressed data directly to thetarcommand. Thetarcommand on the right side of the pipeline looks a little strange, too--instead of a file name after thexvf, there's just a dash. The dash tellstarthat the input is not an actual file on disk, but rather a stream of data from the pipeline. (Note that thegunzipinput file is not deleted when you use the-cflag.)
Here's a third method of extracting the contents of a compressed tar file that's even easier. Remember thezflag with thetarcommand? You can use it to decompress and unbundle a tar file, like this:
tar xvzf something.tar.gz
The end result is exactly the same (the files that were in the compressed tar file are now in your current directory), but this is much easier than issuing multiple commands or writing a messy-lookinggunzip-tarpipeline.
Note that this command will work on all Linux systems, but thezflag fortaris not always available on other flavors of Unix. (However, you can download and compile the source code for the GNU version of thetarcommand. See the note near the beginning of this section about getting the source code for the GNU utilities.)


Sunday, May 1, 2011

Netstat command

Below is some of the example a typically use command syntax for ‘netstat’ to check and show the number of connections a server has. Users can also use ‘man netstat’ command to get detailed netstat help and manual where there are lots of configurable options and flags to get meaningful lists and results.
netstat -na
Display all active Internet connections to the servers and only established connections are included.
netstat -an | grep :80 | sort
Show only active Internet connections to the server at port 80 and sort the results. Useful in detecting single flood by allowing users to recognize many connections coming from one IP.
netstat -n -p|grep SYN_REC | wc -l
Let users know how many active SYNC_REC are occurring and happening on the server. The number should be pretty low, preferably less than 5. On DoS attack incident or mail bombed, the number can jump to twins. However, the value always depends on system, so a high value may be average in another server.
netstat -n -p | grep SYN_REC | sort -u
List out the all IP addresses involved instead of just count.
netstat -n -p | grep SYN_REC | awk '{print $5}' | awk -F: '{print $1}'
List all the unique IP addresses of the node that are sending SYN_REC connection status.
netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
Use netstat command to calculate and count the number of connections each IP address makes to the server.
netstat -anp |grep 'tcp\|udp' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
List count of number of connections the IPs are connected to the server using TCP or UDP protocol.
netstat -ntu | grep ESTAB | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr
Check on ESTABLISHED connections instead of all connections, and displays the connections count for each IP.
netstat -plan|grep :80|awk {'print $5'}|cut -d: -f 1|sort|uniq -c|sort -nk 1
Show and list IP address and its connection count that connect to port 80 on the server. Port 80 is used mainly by HTTP web page request.

Access.log

Out put format of access.log


1. IP address
Every request is accompanied by an IP address. The IP address is where
the response is sent to.2. Machine Identity
The identity of the client's machine (the computer the browser/spider/etc.
software is running on) is unreliable and unlikely to be used on your
server unless it is part of a tightly controlled internal network.
3. User
The user if logged in through Apache authentication.
4. Date and Time
The date, time and time zone between square brackets.
[day/month/year:hour:minute:second zone]
5. Request
The request from the client (browser, spider, etc.) between double quotes.
The request line contains three space-separated items of information.
(I) The request method – GET, POST, or HEAD.
(II) The location of the requested resource and, if applicable, the
query string – the URI of file, script, or other file being requested. If a
query string accompanies the request, it is appended to the URI.
(III) The request protocol – generally HTTP/1.0 or 1.1, but can also
be FTP or other protocol.
6. Status Code
The status code sent back to the client (browser, spider, etc.)7. Size
The size of the object (source code file, image, etc.) sent back to the client
(browser, spider, etc.)
8. Referrer
The referrer information accompanying the request between double
quotes, if available. This is the URL the browser reports it was referred
by, generally the URL of the web page with the link that was clicked or
the URL of the web page that requires an external file (like a CSS or
image file).
9. User-Agent
The user-agent information accompanying the request between double
quotes, if available. This is the identifying information that the client
(browser, spider, etc.) reports about itself