silico.biotoul.fr
 

Linux tips

From silico.biotoul.fr

(Difference between revisions)
Jump to: navigation, search
m (File content, concatenation, split, ... and redirections: cat, split, head, tail, more, less, tac)
m (Paths & I/O & files)
Line 1: Line 1:
= Paths & I/O & files =
= Paths & I/O & files =
 +
 +
== Linux filesystem organization ==
 +
 +
{| style="border-collapse: separate;
 +
          border-spacing: 0;
 +
          border: 1px solid #000;
 +
          padding: 3px"
 +
|-
 +
| / || The root directory.
 +
|}
 +
* /boot This is where the Linux kernel and boot loader files are kept. The kernel is a file called vmlinuz.
 +
/etc The /etc directory contains the configuration files for the system. All of the files in /etc should be text files. Points of interest:
 +
 +
/etc/passwd
 +
    The passwd file contains the essential information for each user. It is here that users are defined.
 +
/etc/fstab
 +
    The fstab file contains a table of devices that get mounted when your system boots. This file defines your disk drives.
 +
/etc/hosts
 +
    This file lists the network host names and IP addresses that are intrinsically known to the system.
 +
/etc/init.d
 +
    This directory contains the scripts that start various system services typically at boot time.
 +
 +
/bin, /usr/bin These two directories contain most of the programs for the system. The /bin directory has the essential programs that the system requires to operate, while /usr/bin contains applications for the system's users.
 +
/sbin, /usr/sbin The sbin directories contain programs for system administration, mostly for use by the superuser.
 +
/usr The /usr directory contains a variety of things that support user applications. Some highlights:
 +
 +
/usr/share/X11
 +
    Support files for the X Windows system
 +
/usr/share/dict
 +
    Dictionaries for the spelling checker. Bet you didn't know that Linux had a spelling checker. See look and ispell.
 +
/usr/share/doc
 +
    Various documentation files in a variety of formats.
 +
/usr/share/man
 +
    The man pages are kept here.
 +
/usr/src
 +
    Source code files. If you installed the kernel source code package, you will find the entire Linux kernel source code here.
 +
 +
/usr/local /usr/local and its subdirectories are used for the installation of software and other files for use on the local machine. What this really means is that software that is not part of the official distribution (which usually goes in /usr/bin) goes here.
 +
 +
When you find interesting programs to install on your system, they should be installed in one of the /usr/local directories. Most often, the directory of choice is /usr/local/bin.
 +
/var The /var directory contains files that change as the system is running. This includes:
 +
 +
/var/log
 +
    Directory that contains log files. These are updated as the system runs. You should view the files in this directory from time to time, to monitor the health of your system.
 +
/var/spool
 +
    This directory is used to hold files that are queued for some process, such as mail messages and print jobs. When a user's mail first arrives on the local system (assuming you have local mail), the messages are first stored in /var/spool/mail
 +
 +
/lib The shared libraries (similar to DLLs in that other operating system) are kept here.
 +
/home /home is where users keep their personal work. In general, this is the only place users are allowed to write files. This keeps things nice and clean :-)
 +
/root This is the superuser's home directory.
 +
/tmp /tmp is a directory in which programs can write their temporary files.
 +
/dev The /dev directory is a special directory, since it does not really contain files in the usual sense. Rather, it contains devices that are available to the system. In Linux (like Unix), devices are treated like files. You can read and write devices as though they were files. For example /dev/fd0 is the first floppy disk drive, /dev/sda (/dev/hda on older systems) is the first IDE hard drive. All the devices that the kernel understands are represented here.
 +
/proc The /proc directory is also special. This directory does not contain files. In fact, this directory does not really exist at all. It is entirely virtual. The /proc directory contains little peep holes into the kernel itself. There are a group of numbered entries in this directory that correspond to all the processes running on the system. In addition, there are a number of named entries that permit access to the current configuration of the system. Many of these entries can be viewed. Try viewing /proc/cpuinfo. This entry will tell you what the kernel thinks of your CPU.
 +
/media,/mnt Finally, we come to /media, a normal directory which is used in a special way. The /media directory is used for mount points. As we learned in the second lesson, the different physical storage devices (like hard disk drives) are attached to the file system tree in various places. This process of attaching a device to the tree is called mounting. For a device to be available, it must first be mounted.
 +
 +
When your system boots, it reads a list of mounting instructions in the file /etc/fstab, which describes which device is mounted at which mount point in the directory tree. This takes care of the hard drives, but you may also have devices that are considered temporary, such as CD-ROMs and floppy disks. Since these are removable, they do not stay mounted all the time. The /media directory is used by the automatic device mounting mechanisms found in modern desktop oriented Linux distributions. On systems that require manual mounting of removable devices, the /mnt directory provides a convenient place for mounting these temporary devices. You will often see the directories /mnt/floppy and /mnt/cdrom. To see what devices and mount points are used, type mount.
 +
== Paths & directories: <tt>pwd, mkdir, rmdir, rm</tt> ==
== Paths & directories: <tt>pwd, mkdir, rmdir, rm</tt> ==
* <tt>pwd</tt> returns current directory
* <tt>pwd</tt> returns current directory

Revision as of 13:15, 5 March 2013

Contents

Paths & I/O & files

Linux filesystem organization

border-spacing: 0; border: 1px solid #000; padding: 3px"
/ The root directory.
  • /boot This is where the Linux kernel and boot loader files are kept. The kernel is a file called vmlinuz.

/etc The /etc directory contains the configuration files for the system. All of the files in /etc should be text files. Points of interest:

/etc/passwd

   The passwd file contains the essential information for each user. It is here that users are defined.

/etc/fstab

   The fstab file contains a table of devices that get mounted when your system boots. This file defines your disk drives.

/etc/hosts

   This file lists the network host names and IP addresses that are intrinsically known to the system.

/etc/init.d

   This directory contains the scripts that start various system services typically at boot time.

/bin, /usr/bin These two directories contain most of the programs for the system. The /bin directory has the essential programs that the system requires to operate, while /usr/bin contains applications for the system's users. /sbin, /usr/sbin The sbin directories contain programs for system administration, mostly for use by the superuser. /usr The /usr directory contains a variety of things that support user applications. Some highlights:

/usr/share/X11

   Support files for the X Windows system

/usr/share/dict

   Dictionaries for the spelling checker. Bet you didn't know that Linux had a spelling checker. See look and ispell.

/usr/share/doc

   Various documentation files in a variety of formats.

/usr/share/man

   The man pages are kept here.

/usr/src

   Source code files. If you installed the kernel source code package, you will find the entire Linux kernel source code here.

/usr/local /usr/local and its subdirectories are used for the installation of software and other files for use on the local machine. What this really means is that software that is not part of the official distribution (which usually goes in /usr/bin) goes here.

When you find interesting programs to install on your system, they should be installed in one of the /usr/local directories. Most often, the directory of choice is /usr/local/bin. /var The /var directory contains files that change as the system is running. This includes:

/var/log

   Directory that contains log files. These are updated as the system runs. You should view the files in this directory from time to time, to monitor the health of your system.

/var/spool

   This directory is used to hold files that are queued for some process, such as mail messages and print jobs. When a user's mail first arrives on the local system (assuming you have local mail), the messages are first stored in /var/spool/mail

/lib The shared libraries (similar to DLLs in that other operating system) are kept here. /home /home is where users keep their personal work. In general, this is the only place users are allowed to write files. This keeps things nice and clean :-) /root This is the superuser's home directory. /tmp /tmp is a directory in which programs can write their temporary files. /dev The /dev directory is a special directory, since it does not really contain files in the usual sense. Rather, it contains devices that are available to the system. In Linux (like Unix), devices are treated like files. You can read and write devices as though they were files. For example /dev/fd0 is the first floppy disk drive, /dev/sda (/dev/hda on older systems) is the first IDE hard drive. All the devices that the kernel understands are represented here. /proc The /proc directory is also special. This directory does not contain files. In fact, this directory does not really exist at all. It is entirely virtual. The /proc directory contains little peep holes into the kernel itself. There are a group of numbered entries in this directory that correspond to all the processes running on the system. In addition, there are a number of named entries that permit access to the current configuration of the system. Many of these entries can be viewed. Try viewing /proc/cpuinfo. This entry will tell you what the kernel thinks of your CPU. /media,/mnt Finally, we come to /media, a normal directory which is used in a special way. The /media directory is used for mount points. As we learned in the second lesson, the different physical storage devices (like hard disk drives) are attached to the file system tree in various places. This process of attaching a device to the tree is called mounting. For a device to be available, it must first be mounted.

When your system boots, it reads a list of mounting instructions in the file /etc/fstab, which describes which device is mounted at which mount point in the directory tree. This takes care of the hard drives, but you may also have devices that are considered temporary, such as CD-ROMs and floppy disks. Since these are removable, they do not stay mounted all the time. The /media directory is used by the automatic device mounting mechanisms found in modern desktop oriented Linux distributions. On systems that require manual mounting of removable devices, the /mnt directory provides a convenient place for mounting these temporary devices. You will often see the directories /mnt/floppy and /mnt/cdrom. To see what devices and mount points are used, type mount.

Paths & directories: pwd, mkdir, rmdir, rm

  • pwd returns current directory
  • relative to current directory: e.g. ls subdir/subsubdir or ls ../whatever/
  • absolute ls ~user/path or ls /home/user/path
  • mkdir: create directory. e.g. mkdir ~/newdir or with subdirs mkdir -p ~/new/newsub/newsubsub
  • rmdir dirname or if not empty rm -fr dirname

Permissions: chown, chgrp, chmod

$ ls -l /home
drwxr-x---  69 barriot    gsi   4.0K Mar  5 12:09 barriot
drwx------   2 root       root   16K Jul 12  2010 lost+found
drwxr-xr-x  36 micas      stage 4.0K Jul 31  2012 micas
...
 
[barriot@gamborimbo ~]$ ls -lh Documents/TEACHING/2012-2013/M1-MABS/Graph/TP3-igraph.layout/
total 80K
drwxr-xr-x 1 barriot gsi 4.0K Mar 14  2012 HDE.old
-rw-r--r-- 1 barriot gsi  24K Mar 14  2012 91347.nwk
-rw-r--r-- 1 barriot gsi  942 Mar  1 16:02 Cleandb_Luca_1_S_1_1_65_Iso_Tr_1-CC1.cod
-rw-r--r-- 1 barriot gsi  28K Sep  7  2010 Cleandb_Luca_1_S_1_1_65_Iso_Tr_1-CC1.gr
-rw-r--r-- 1 barriot gsi 2.3K Sep  7  2010 Cleandb_Luca_1_S_1_1_65_Iso_Tr_1-CC1.tgr
-rw-r--r-- 1 barriot gsi 4.7K Mar  5 11:42 cmds.R
-rw-r--r-- 1 barriot gsi  871 Mar 14  2012 sample_tree_with_branchlengths.nwk
-rwxr-xr-x 1 barriot gsi  670 Mar 14  2012 drawTree.py
-rw-r--r-- 1 barriot gsi 5.6K Feb 27 16:57 Tree.py

First character corresponds to file type. d for directory, - for a regular file, ... Then by 3 for the owner (user), the group and the others.

For a regular file :

  • r for permission to read
  • w for permission to modify
  • x for being able to execute the file (binary executable or script)

For a directory :

  • r to be able to read the content (list files in the directory)
  • w to be able to add or remove files
  • x to be able to pass through that directory, i.e. cd to that dir or a subdir

Modify ownership of a file or directory :

# change owner
chown newuser file
# recursive
chown -R newuser directory
# change group
chgrp newgroup filename
# change both
chown newuser.newgroup filename

Modify permissions:

# numeric notation: r=4, w=2, x=1, thus for rwx-r-x---
chmod 760 file
# recursively on a sub directory
chmod -R 760 dirname
# symbolic notation:
chmod u=rwx,g=rx,o= filename
# add execute permission for all:
chmod a+x filename
# revoke write permission for others:
chmod o-w filename

File info & type: stat, file

[barriot@gamborimbo ~]$ stat /home/barriot
  File: `/home/barriot'
  Size: 12288     	Blocks: 24         IO Block: 4096   directory
Device: fd02h/64770d	Inode: 1048577     Links: 119
Access: (0755/drwxr-xr-x)  Uid: (  500/ barriot)   Gid: (  501/     gsi)
Access: 2013-03-05 10:39:08.927051453 +0100
Modify: 2013-03-05 10:39:00.240074369 +0100
Change: 2013-03-05 10:39:00.240074369 +0100
 Birth: -
[barriot@gamborimbo ~]$ stat .bashrc
  File: `.bashrc'
  Size: 517       	Blocks: 8          IO Block: 4096   regular file
Device: fd02h/64770d	Inode: 1052239     Links: 1
Access: (0755/-rwxr-xr-x)  Uid: (  500/ barriot)   Gid: (  501/     gsi)
Access: 2013-03-02 16:04:19.268619379 +0100
Modify: 2012-10-12 17:24:24.818899216 +0200
Change: 2012-11-18 23:25:18.869870338 +0100
 Birth: -
[barriot@gamborimbo ~]$ file /home/barriot
/home/barriot: directory
[barriot@gamborimbo ~]$ file .bashrc
.bashrc: ASCII text

File content, concatenation, split, ... and redirections: cat, split, head, tail, more, less, tac

# display content
cat somefile.txt
# concatenate 2 or more files
cat file_1.txt file_2.txt
cat *.txt
# redirect to a file (if file exists it will be overwritten otherwise it gets created)
cat file_1.txt file_2.txt > result.txt
# redirect to a file (if file exists it will be appended at the end otherwise it gets created)
cat others*.txt >> result.txt
 
# split a file into smaller parts
## by file size (1kb)
split --bytes 1024 big.file
split -b 1024 big.file
## by number of lines per output files
split --lines 100 big.text.file.txt
split -l 100 big.text.file.txt
## by number of output files
split --number 10 big.file
split -n 10 big.file
## specify output files prefix and numbered numerically (3 digits)
split -n 100 -a 3 -d big.file part_ 
split -n 100 --suffix-length 3 --numeric-suffixes big.file part_


redirection, head, tail, more, less, tac, sed, grep, cut, find, wc, sort

Processes

ps, jobs, Ctrl-C/Z/D, top, kill, killall, nohup, disown, &

shell

variables, test, $?, for, if, function, $(cmd), && ||

archive

tar, bzip, gzip, rsync