Wednesday, May 25, 2011

gzip

Compressing Multiple Files

You can concatenate multiple compressed files. When you do so, gunzip (or gzip -d) extracts all files in the compressed file as a single file. For example:
gzip -c file1  > catfiles.gz
gzip -c file2 >> catfiles.gz
After creating the compressed file catfiles.gz, the command:
gunzip -c catfiles.gz
is equivalent to
cat file1 file2
If one of the files in such a .gz file is damaged or corrupt, the other files can still be recovered (if the damaged or corrupt member is removed).
You can improve the level of compression achieved by compressing all the files at once rather than compressing them individually and then concatenating the results. For example:
cat file1 file2 | gzip > catfiles.gz
yields better compression than:
gzip -c file1 file2 > catfiles.gz
You can recompress concatenated files to get better compression with a command like:
gzip -cd old.gz | gzip > new.gz
When a compressed file contains several individual files, the uncompressed size and CRC reported by the --list option are for the last member only. To get the uncompressed size for all members, use:
gzip -cd file.gz | wc -c
Multiple compressed files can be concatenated. In this case, gunzip will extract all members at once. For example:

gzip -c file1 > foo.gz
gzip -c file2 >> foo.gz
Then

gunzip -c foo
is equivalent to

cat file1 file2
In case of damage to one member of a .gz file, other members can still be recovered (if the damaged member is removed). However, you can get better compression by compressing all members at once:

cat file1 file2 | gzip > foo.gz
compresses better than

gzip -c file1 file2 > foo.gz
If you want to recompress concatenated files to get better compression, do:

gzip -cd old.gz | gzip > new.gz
If a compressed file consists of several members, the uncompressed size and CRC reported by the --list option applies to the last member only. If you need the uncompressed size for all members, you can use:

gzip -cd file.gz | wc -c
If you wish to create a single archive file with multiple members so that members can later be extracted independently, use an archiver such as tar or zip. GNU tar supports the -z option to invoke gzip transparently. gzip is designed as a complement to tar, not as a 
One other useful option is the-rflag, which tellsgzipandgunzipto recursively compress or decompress all files in the current directory and any subdirectories. (Even with the-rflag,gzipstill compresses one file at a time.) Here are some examples:
gzip -r somedirZip all files in thesomedirdirectory.
gunzip -r somedirUnzip all files in thesomedirdirectory.

Handling Compressed Archives
It's common to applygzipto a tar file, which is why you see files with names likesomething.tar.gzon Linux systems. When you want to extract the contents of a gzipped tar file, you have several choices. The first is to usegunzipfollowed bytar, like this:
gunzip something.tar.gz
tar xvf something.tar

Or you could do it all in one command, like this:
gunzip -c something.tar.gz | tar xvf -
The-cflag tellsgunzipto decompress the file, but instead of creating asomething.tarfile, it pipes the decompressed data directly to thetarcommand. Thetarcommand on the right side of the pipeline looks a little strange, too--instead of a file name after thexvf, there's just a dash. The dash tellstarthat the input is not an actual file on disk, but rather a stream of data from the pipeline. (Note that thegunzipinput file is not deleted when you use the-cflag.)
Here's a third method of extracting the contents of a compressed tar file that's even easier. Remember thezflag with thetarcommand? You can use it to decompress and unbundle a tar file, like this:
tar xvzf something.tar.gz
The end result is exactly the same (the files that were in the compressed tar file are now in your current directory), but this is much easier than issuing multiple commands or writing a messy-lookinggunzip-tarpipeline.
Note that this command will work on all Linux systems, but thezflag fortaris not always available on other flavors of Unix. (However, you can download and compile the source code for the GNU version of thetarcommand. See the note near the beginning of this section about getting the source code for the GNU utilities.)


Sunday, May 1, 2011

Netstat command

Below is some of the example a typically use command syntax for ‘netstat’ to check and show the number of connections a server has. Users can also use ‘man netstat’ command to get detailed netstat help and manual where there are lots of configurable options and flags to get meaningful lists and results.
netstat -na
Display all active Internet connections to the servers and only established connections are included.
netstat -an | grep :80 | sort
Show only active Internet connections to the server at port 80 and sort the results. Useful in detecting single flood by allowing users to recognize many connections coming from one IP.
netstat -n -p|grep SYN_REC | wc -l
Let users know how many active SYNC_REC are occurring and happening on the server. The number should be pretty low, preferably less than 5. On DoS attack incident or mail bombed, the number can jump to twins. However, the value always depends on system, so a high value may be average in another server.
netstat -n -p | grep SYN_REC | sort -u
List out the all IP addresses involved instead of just count.
netstat -n -p | grep SYN_REC | awk '{print $5}' | awk -F: '{print $1}'
List all the unique IP addresses of the node that are sending SYN_REC connection status.
netstat -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
Use netstat command to calculate and count the number of connections each IP address makes to the server.
netstat -anp |grep 'tcp\|udp' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n
List count of number of connections the IPs are connected to the server using TCP or UDP protocol.
netstat -ntu | grep ESTAB | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr
Check on ESTABLISHED connections instead of all connections, and displays the connections count for each IP.
netstat -plan|grep :80|awk {'print $5'}|cut -d: -f 1|sort|uniq -c|sort -nk 1
Show and list IP address and its connection count that connect to port 80 on the server. Port 80 is used mainly by HTTP web page request.

Access.log

Out put format of access.log


1. IP address
Every request is accompanied by an IP address. The IP address is where
the response is sent to.2. Machine Identity
The identity of the client's machine (the computer the browser/spider/etc.
software is running on) is unreliable and unlikely to be used on your
server unless it is part of a tightly controlled internal network.
3. User
The user if logged in through Apache authentication.
4. Date and Time
The date, time and time zone between square brackets.
[day/month/year:hour:minute:second zone]
5. Request
The request from the client (browser, spider, etc.) between double quotes.
The request line contains three space-separated items of information.
(I) The request method – GET, POST, or HEAD.
(II) The location of the requested resource and, if applicable, the
query string – the URI of file, script, or other file being requested. If a
query string accompanies the request, it is appended to the URI.
(III) The request protocol – generally HTTP/1.0 or 1.1, but can also
be FTP or other protocol.
6. Status Code
The status code sent back to the client (browser, spider, etc.)7. Size
The size of the object (source code file, image, etc.) sent back to the client
(browser, spider, etc.)
8. Referrer
The referrer information accompanying the request between double
quotes, if available. This is the URL the browser reports it was referred
by, generally the URL of the web page with the link that was clicked or
the URL of the web page that requires an external file (like a CSS or
image file).
9. User-Agent
The user-agent information accompanying the request between double
quotes, if available. This is the identifying information that the client
(browser, spider, etc.) reports about itself

Tuesday, April 5, 2011

Find command

Find command for file larger than 200mb
find / -type f -size +20000k -exec ls -lh {} \; | awk '{ print $9 ": " $5 }'

find . -size +20000k -exec du -h {} \;
Sort-------------------------
find / -type f -size +20000k -exec ls -lh {} \; 2> /dev/null | awk '{ print $NF ": " $5 }' | sort -nk 2,2

sudo find / -type f -size +100000k -exec ls -lh {} \; | awk '{ print $8 ": " $5 }'

FTP,SCP command

ftp  <remote hostname>
It will ask for Username and password of remote system
ftp> put  <local to remote file>
ftp>get <remote to local file>
ft>bye  (exit from ftp)
Goodbye message

SCP command

Example of a scp download:

# scp user@machine:/path/to/remote/file /local/directory

Example of a scp upload:

# scp /local/file(s)/to/transfer user@machine:/remote/dir/to/transfer/to
Example

scp user@machine:/path/to/remote/file /local/directory

# scp user@machine:/path/to/remote/file /local/directory

Example of a scp upload:

# scp /local/file(s)/to/transfer user@machine:/remote/dir/to/transfer/to
Example

scp user@machine:/path/to/remote/file /local/directory

Sunday, March 20, 2011

LDAP Authentication in WebSphere

1. A request arrives on an input channel (for example, Web or EJB).
2. The authentication data is passed through the authentication modules.
3. This flow highlights the WebSphere Application Server default trust
association interceptor (TAI) for LTPA token. LTPA is the application server’s
default and the recommended trust token implementation. Trust is asserted at
this stage of the processing. If the token is determined valid, the user
information that it contains is trusted and the identity of the user is asserted.
4. After token processing, the user credentials are

rebuilt from the information that is retrieved from the token.
5. User credentials are created based on user information retrieved from the
registry.
6. The credential is forwarded as the request is processed by different
architectural tiers of the application server.

How to Turnoff Security for WebSphere Admin console

If you are unable to authenticate, you can turn off security to help
troubleshoot the problem by running the following wsadmin commands
(command in jacl format). You must run this command locally; it cannot be
executed remotely.
Executed from /bin directory where is the profile’s install
directory.
wsadmin -conntype NONE
WASX7357I: By request, this scripting client is not connected to any
server process. Certain configuration and application operations
will be available in local mode.
WASX7029I: For help, enter: "$Help help"
wsadmin>securityoff
LOCAL OS security is off now but you need to restart server1 to make
it affected.
wsadmin>$AdminConfig save
wsadmin>exit
----------------------------------------------

WAS_INSTALL_DIR/profiles/<profileName>/config/cells/<cellName>/
and open the file security.xml. Search for a tag name
<security:Security.
This would be the first tag in security.xml file. In this tag we can see an attribute called
enabled="true"
Just change the value to false. Save the file and restart the server. Security will be disabled on the server.
---------------------------------------------------------------------------------------------------------------------------------

WAS Admin Security

WebSphere Security.

Configuring security with scripting

You can configure security with scripting and the wsadmin tool.

Before you begin

Before starting this task, the wsadmin tool must be running. See the Starting the wsadmin scripting client article for more information.

About this task

If you enable security for a WebSphere Application Server cell, supply authentication information to communicate with servers. The sas.client.props and the soap.client.props files are located in the following properties directory for each WebSphere Application Server profile:

Procedure

  • The nature of the properties file updates required for running in secure mode depend on whether you connect with a Remote Method Invocation (RMI) connector, or a SOAP connector:
    • If you use a Remote Method Invocation (RMI) connector, set the following properties in thesas.client.props file with the appropriate values:
o                     com.ibm.CORBA.loginUserid=
com.ibm.CORBA.loginPassword=
Also, set the following property:
com.ibm.CORBA.loginSource=properties
The default value for this property is prompt in the sas.client.props file. If you leave the default value, a dialog box appears with a password prompt. If the script is running unattended, it appears to hang.
    • If you use a SOAP connector, set the following properties in the soap.client.props file with the appropriate values:
o                     com.ibm.SOAP.securityEnabled=true
o                     com.ibm.SOAP.loginUserid=
com.ibm.SOAP.loginPassword=
Optionally, set the following property:
com.ibm.SOAP.loginSource=none
The default value for this property is prompt in the soap.client.props file. If you leave the default value, a dialog box appears with a password prompt. If the script is running unattended, it appears to hang.
  • Specify user and password information. Choose one of the following methods:
    • Specify user name and password on a command line, using the -user and -passwordcommands. For example:
wsadmin -conntype RMI -port 2809 -user u1 -password secret1
    • Specify user name and password in the sas.client.props file for a RMI connector or thesoap.client.props file for a SOAP connector.
If you specify user and password information on a command line and in the sas.client.props file or thesoap.client.props file, the command line information overrides the information in the props file.
Note: The use of -password option may result in security exposure as the password information becomes visible to the system status program such as ps command which can be invoked by other user to display all the running processes. Do not use this option if security exposure is a concern. Instead, specify user and password information in the soap.client.props file for SOAP connector or sas.client.props file for RMI connector. The soap.client.props and sas.client.props files are located in the properties directory of your WebSphere Application Server profile
To run any command without exposing the password in the command line, use the below format command for any operation through shell.
WAS-TEST:/opt/IBM/WebSphere/AppServer/profiles/Dmgr01/bin #./stopManager.sh -conntype SOAP
ADMU0116I: Tool information is being logged in file
/opt/IBM/WebSphere/AppServer/profiles/Dmgr01/logs/dmgr/stopServer.log
ADMU0128I: Starting tool with the Dmgr01 profile
ADMU3100I: Reading configuration for server: dmgr
ADMU3201I: Server stop request issued. Waiting for stop status.
ADMU4000I: Server dmgr stop completed.
Example: Enabling and disabling Java 2 security using wsadmin
An example of enabling and disabling Java 2 security follows:
  • Identify the security configuration object and assign it to the security variable:
Using Jacl:
set security [$AdminConfig list Security]
An example of this output follows:
(cells/mycell:security.xml#Security_1)
[Version 5.1 and later]Using Jython:
security = AdminConfig.list('Security')
print security
  • Modify the enforceJava2Security attribute.
To enable Java 2 security:
Using Jacl:
$AdminConfig modify $security {{enforceJava2Security true}}
[Version 5.1 and later]Using Jython:
AdminConfig.modify(security, [['enforceJava2Security', 'true']])
To disable Java 2 security:
Using Jacl:
$AdminConfig modify $security {{enforceJava2Security false}}
[Version 5.1 and later]Using Jython:
AdminConfig.modify(security, [['enforceJava2Security', 'false']])
  • Save the changes with the following command:
Using Jacl:
$AdminConfig save
[Version 5.1 and later]Using Jython:
AdminConfig.save()

IF the security has to be disabled through wsadmin:

Enforce Java 2 Security
Specifies whether to enable or disable Java 2 Security permission checking. By default, Java 2 security is disabled. However, if you enabled global security, this automatically enables Java 2 security. You can choose to disable Java 2 security, even when global security is enabled.
When Java 2 Security is enabled and if an application requires more Java 2 security permissions then are granted in the default policy, then the application might fail to run properly until the required permissions are granted in either the app.policy file or the was.policy file of the application. AccessControl exceptions are generated by applications that do not have all the required permissions. Consult the InfoCenter and review the Java 2 Security and Dynamic Policy sections if you are unfamiliar with Java 2 security.
If your server does not restart after you enable global security, you can disable security. Go to your${was_install_root}\bin directory.
Excecute the command wsadmin -conntype NONE.
At the wsadmin> prompt, enter securityoff.
Type exit to get back to a command prompt.
Now you should be able to start the server again, with security disabled. This enables you to check what might not be set correctly through the administrative console.
Data type
Boolean
Default
Disabled
Range
Enabled or Disabled