Wonderful Find Command

"find" is a versatile tool which can be used to locate files and directories satisfying different user criteria. But the sheer number of options for this command line tool makes it at the same time both powerful and encumbering for the user.

Here are a few combinations which one can use to get useful results using find command.

* Find all HTML files starting with letter 'a' in your current directory (Case sensitive):

Code: find . -name a\*.html

* Find files which are larger than 5 MB in size:

Code: find . -size 5000k -type f

Here the ' ' in ' 5000k' indicates greater than and k is kilobytes. And the dot '.' indicates the current directory. The -type option can take any of the following values:
f - file
d - directory
l - symbolic link
c - character
p - named pipe (FIFO)
s - socket
b - block device

* Find all empty files in your directory:

Code: find . -size 0c -type f

* Find is very powerful in that you can combine it with other commands. For example, to find all empty files in the current directory and delete them, do the following:

Code: find . -empty -maxdepth 1 -exec rm {} \;

* To search for a html file having the text 'Web sites' in it, you can combine find with grep as follows:

Code: find . -type f -iname \*.html -exec grep -s "Web sites" {} \;

* Compress log files on an individual basis:

Code: find /var -iname \*.log -exec bzip {} \;

* Find all files which belong to user lal and change its ownership to ravi:

Code: find / -user lal -exec chown ravi {} \;

* xargs command can also be used instead of the -exec option as follows:

Code: find /var -iname \*.log | xargs bzip –

* Find all files which do not belong to any user:

Code: find . –nouser

* find files of size between 700k and 1000k:

Code: find . \( -size 700k -and -size -1000k \)


* How about getting a formatted output of the above command with the size of each file listed ?:

Code: find . \( -size 700k -and -size -1000k \) -exec du -Hs {} \; 2>/dev/null


* You can also limit your search by file system type. For example, to restrict search to files residing only in the NTFS and VFAT filesystem, do the following:

Code: find / -maxdepth 2 \( -fstype vfat -or -fstype ntfs \) 2> /dev/null

Regards
Nobs

How to Reset a Linux Box root password.

If the linux box protected with root password only.

Do a hard reboot on the machine and click on ‘e’ button to edit the boot like when you get the grub menu. Scrol down to the line which starts with kernel and hit ‘e’ again to edit and add ‘single’ (with out quotes) at the end and hit Enter. Now use ‘b’ to boot then you will get single user mode from where you can change the server root password.

If the linux box is protected with a password for single user mode.

Few system require root password to boot into ‘single mode’ in that case try to edit the gurb boot line and add ‘init=/bin/bash’ at the end of kernel line. You will be redirected to a bash shell instead of init.

Now you may want to mount the root paritition with read/write permission.

mount –no remount,rw /

Then reset the password.

If the Grub is protected with a root password, then try booting from a LiveCD and open a root shell. Execute fdisk –l to show the available disk partition. Mount the root paritition to /mnt

mount -o,rw /dev/hda1 /mnt

Make sure that you mount a root partition. Next change this partition as your root directory using following command

chroot /mnt

Now try to change the root password using passwd command.

Regards
Nobs

Increasing Mysql Packet Size

A communication packet is a single SQL statement sent to the MySQL server, a single row that is sent to the client, or a binary log event sent from a master replication server to a slave.

The largest possible packet that can be transmitted to or from a MySQL 5.1 server or client is 1GB.

When a MySQL client or the mysqld server receives a packet bigger than max_allowed_packet bytes, it issues a Packet too large error and closes the connection. With some clients, you may also get a Lost connection to MySQL server during query error if the communication packet is too large.

So if you want to handle big packets, you must increase this variable both in the client and in the server.

The server's default max_allowed_packet value is 1MB. You can increase this if the server needs to handle big queries. For example, to set the variable to 16MB, start the server like this:

# mysqld --max_allowed_packet=16M

You can also use my.cnf to set max_allowed_packet. For example, to set the size for the server to 16MB, add the following lines in my.cnf file:

[mysqld]
max_allowed_packet=16M

You can also get strange problems with large packets if you are using large BLOB values but have not given mysqld access to enough memory to handle the query. If you suspect this is the case, try adding ulimit -d 256000 to the beginning of the mysqld_safe script and restarting mysqld.

Wonderful lsof Command

lsof is a tool for locating open files. What makes this especially useful is that in Linux, everything is treated as a file: pipes, directories, devices, inodes, sockets and so on.

lsof (no options) will list all files opened by any processes currently running. To restrict this to processes owned by username, use lsof -u username. Here's some sample output:

COMMAND PID USER FD TYPE DEVICE SIZE NODE NAME
sshd 2354 nobs mem REG 254,0 14880 105723 /lib/libcap.so.1.10
sshd 2354 nobs DEL REG 0,8 127123574 /dev/zero
bash 2363 nobs cwd DIR 254,4 20480 7274497 /home/nobs
bash 2363 nobs txt REG 254,0 769368 4126 /bin/bash
bash 2363 nobs mem REG 254,0 97928 105698 /lib/ld-2.3.6.so

The FD column shows file descriptor information, or identifies other types of file. Here, cwd indicates the current working directory, and txt indicates program text. The TYPE column has filetype info (REG indicates a regular file). The NODE column may be useful if you're trying to recover a deleted file. See the man page for a full explanation of the output.

lsof filename shows which processes have files of this name open. lsof +D /directory will show processes which have files in this directory open. You can use this if you're trying to unmount a filesystem but getting an 'in use' error, to find the processes using files on that File System, and kill them as required.

lsof -c processname will show all processes beginning with processname that have files open; lsof +p PID does the same thing for a process ID. Using lsof -i will get you information about IP sockets. Check out the man page for more detail and for the many other available options. lsof is really a cool one to play around.

Cheers
Nobs

Recovering Deleted Files With lsof

One of the more neat things you can do with the versatile utility lsof is use it to recover a file you've just accidentally deleted.

A file in Linux is a pointer to an inode, which contains the file data (permissions, owner and where its actual content lives on the disk). Deleting the file removes the link, but not the inode itself – if another process has it open, the inode isn't released for writing until that process is done with it.

To try this out, create a test text file, save it and then type less test.txt. Open another terminal window, and type rm testing.txt. If you try ls testing.txt you'll get an error message. But! less still has a reference to the file. So:

# lsof | grep testing.txt
less 4607 juliet 4r REG 254,4 21
8880214 /home/juliet/testing.txt (deleted)

The important columns are the second one, which gives you the PID of the process that has the file open (4607), and the fourth one, which gives you the file descriptor (4). Now, we go look in /proc, where there will still be a reference to the inode, from which you can copy the file back out:

# ls -l /proc/4607/fd/4
lr-x------ 1 juliet juliet 64 Apr 7 03:19
/proc/4607/fd/4 -> /home/juliet/testing.txt (deleted)
# cp /proc/4607/fd/4 testing.txt.bk

Note: don't use the -a flag with cp, as this will copy the (broken) symbolic link, rather than the actual file contents.

Now check the file to make sure you've got what you think you have, and you're done!

Cheers
Nobs

Difference Between Soft Link and Hard Link

Hard Links :

1. All Links have same inode number.

2.ls -l command shows all the links with the link column(Second) shows No. of links.

3. Links have actual file contents

4.Removing any link ,just reduces the link count , but doesn’t affect other links.

Soft Links(Symbolic Links) :

1.Links have different inode numbers.

2. ls -l command shows all links with second column value 1 and the link points to original file.

3. Link has the path for original file and not the contents.

4.Removing soft link doesn’t affect anything but removing original file ,the link becomes “dangling” link which points to nonexistant file.

Regards
Nobs

Migrating LVM Volumes Over Network (using snapshots)

We run a big share of Xen virtual servers spanned over multiple servers and if you want to use the full or best capability of Xen, I would suggest LVM (Logical Volume Manager), it makes life a lot easier, especially for those who do not run a RAID setup (We run RAID10 on all VM nodes) as you can split the partition over multiple hard drives. I’m not going to cover setting up the LVM as there are loads of tutorials on how to do that but I will rather cover the best way to migrate a LVM volume.

First, we will need to create a snapshot of the LVM volume as we cannot create an image of the live version, we run the following line:

lvcreate -L20G -s -n storageLV_s /dev/vGroup/storageLV

The 20G part is the size of the snapshot LVM, I would suggest looking up the size of the real original LV and making it the same, you can find out the size of the LV by using this command:
lvdisplay /dev/vGroup/storageLV —
There will be a “LV Size” field, get it from there and put it in the command, the -n switch is for the name, usually I name them the same as the LV with a trailing _s for snapshot, the last argument is simply the real LV that we want to make a snapshot of.

Afterwards, we will use dd in different way, usually if you use dd in one line, it’s either reading or it’s either writing which makes it crawl, to bypass this, we will read the LV and pipe it to one that writes so the minimum speed is the fastest speed of the slowest hard drive (I could re-phrase that but it’s 11:10 PM!) — To speed it up a bit more, we used a block size of 64K.

dd if=/dev/vGroup/storageLV_s conv=noerror,sync bs=64k | dd of=/migrate/storageLV_s.dd bs=64k


I won’t cover the file transfer process as there are multiple methods, if you want to use SCP, I would suggest disabling encryption or anything as it really slows it down, our node usually has httpd installed on them so I simply changed the configuration to listen on a different port (for security) and changed the DocumentRoot to /migrate

Once you got your file on the server, you’ll need to re-create the LV on the target server, you’ll need to run this

lvcreate -L20G -n storageLV vGroup

You’ll have to keep the same size, bring the same name (this time without a trailing _s as it won’t be a snapshot) and the volume group at the end.

The last step is to actually restore the image using dd, again using our block-size & pipe tweak for better performance.

dd if=/migrate/storageLV_s.dd conv=noerror,sync bs=64k | dd of=/dev/vGroup/storageLV bs=64k


I have migrated around 16 LVs with this method without any problems, 13 of them were 20G each, 2 40G and 1 75G — So far every part is fast however I have to admit that the slowest part was the file transfer, I would suggest using a Gbit crossover or even better if you have a Gbit switch, if you don’t but you’re right next to the server, might consider using a spare USB 2.0 HDD as they are much faster compared to 100mbps (USB2.0 is around 480Mbps).

Thanks
Nobs