Files
ls --- lists your files
ls -l --- lists your files in 'long format', which contains lots of useful information, e.g. the exact size of the file, who owns the file and who has the right to look at it, and when it was last modified.
ls -a --- lists all files, including the ones whose filenames begin in a dot, which you do not always want to see.
There are many more options, for example to list files by size, by date, recursively etc.
more filename --- shows the first part of a file, just as much as will fit on one screen. Just hit the space bar to see more or q to quit. You can use /pattern to search for a pattern.
emacs filename --- is an editor that lets you create and edit a file.
mv filename1 filename2 --- moves a file (i.e. gives it a different name, or moves it into a different directory (see below)
cp filename1 filename2 --- copies a file
rm filename --- removes a file. It is wise to use the option rm -i, which will ask you for confirmation before actually deleting anything. You can make this your default by making an alias in your .cshrc file.
diff filename1 filename2 --- compares files, and shows where they differ
wc filename --- tells you how many lines, words, and characters there are in a file
chmod options filename --- lets you change the read, write, and execute permissions on your files. The default is that only you can look at them and change them, but you may sometimes want to change these permissions. For example, chmod o+r filename will make the file readable for everyone, and chmod o-r filename will make it unreadable for others again. Note that for someone to be able to actually look at the file the directories it is in need to be at least executable. See help protection for more details.
File Compression
gzip filename --- compresses files, so that they take up much less space. Usually text files compress to about half their original size, but it depends very much on the size of the file and the nature of the contents. There are other tools for this purpose, too (e.g. compress), but gzip usually gives the highest compression rate. Gzip produces files with the ending '.gz' appended to the original filename.
gunzip filename --- uncompresses files compressed by gzip.
gzcat filename --- lets you look at a gzipped file without actually having to gunzip it (same as gunzip -c). You can even print it directly, using gzcat filename | lpr
printing
lpr filename --- print. Use the -P option to specify the printer name if you want to use a printer other than your default printer. For example, if you want to print double-sided, use 'lpr -Pvalkyr-d', or if you're at CSLI, you may want to use 'lpr -Pcord115-d'. See 'help printers' for more information about printers and their locations.
lpq --- check out the printer queue, e.g. to get the number needed for removal, or to see how many other files will be printed before yours will come out
lprm jobnumber --- remove something from the printer queue. You can find the job number by using lpq. Theoretically you also have to specify a printer name, but this isn't necessary as long as you use your default printer in the department.
genscript --- converts plain text files into postscript for printing, and gives you some options for formatting. Consider making an alias like alias ecop 'genscript -2 -r \!* | lpr -h -Pvalkyr' to print two pages on one piece of paper.
dvips filename --- print .dvi files (i.e. files produced by LaTeX). You can use dviselect to print only selected pages. See the LaTeX page for more information about how to save paper when printing drafts.
Thursday, February 17, 2011
Useful UNIX Utilities for Shell Programmers
Useful UNIX Utilities for Shell Programmers
at—at, batch—execute commands at a later time
awk—pattern scanning and processing language
banner—make posters
basename—with a directory name delivers portions of the pathname
bc—processes precision arithmetic
bdiff—compares two big files
cal—displays a calendar
cat—concatenates and displays files
chmod—change the permissions mode of a file
chown—changes owner of file
clear—clears the terminal screen
cmp—compares two files
compress—compress, uncompress, zcat compress, uncompress files, or display expanded files
cp—copies files
cpio—copy file archives in and out
cron—the clock daemon
crypt—encodes or decodes a file
cut—removes selected fields or characters from each line of a file
date—displays the date and time or sets the date
diff—compares two files for differences diff [–bitw] [–c | –Cn
du—summarizes disk usage
echo—echoes arguments
egrep—searches a file for a pattern using full regular expressions
expr—evaluates arguments as an expression
fgrep—search a file for a character string
file—determines the type of a file by looking at its contents
find—finds files
finger—displays information about local and remote users
fmt—simple text formatters
fold—folds long lines
ftp—file transfer program
getopt(s)—parses command line options
grep—searches a file for a pattern
groups—prints group membership of user
id—prints the username, user ID, group name and group ID
jsh—the standard, job control shell
line—reads one line
logname—gets the name of the user running the process
lp—sends output to a printer (AT&T)
lpr—sends output to a printer (UCB)
lpstat—print information about the status of the LP print service (AT&T)
lpq—print information about the status of the printer (UCB)
ls—lists contents of directory
mail—mail, rmail—read mail or send mail to users
mailx—interactive message processing system
make—maintains, updates, and regenerates groups of related programs and files
mesg—permits or denies messages resulting from the write command
mkdir—creates a directory
more—browse or page through a text file
mv—move or rename files
nawk—pattern scanning and processing language
newgrp—log in to a new group
news—prints news items
nice—runs a command at low priority
nohup—makes commands immune to hangups and quits
od—octal dump
pack—pack, pcat, unpack—compresses and expands files
passwd—changes the login password and password attributes
paste—merges same lines of several files or subsequent lines of one file
pcat—(see pack)
pg—displays files a page at a time
pr—prints files
ps—reports process status
pwd—displays the present working directory name
rcp—remote file copy
rlogin—remote login
rm—removes files from directories
rmdir—removes a directory
rsh—starts a remote shell
ruptime—shows the host status of local machines
rwho—who is logged in on local machines
script—creates a typescript of a terminal session
sed—streamlined editor
size—prints section sizes in bytes of object files
sleep—suspends execution for some number of seconds
sort—sort and/or merge files
spell—finds spelling errors
split—splits a file into pieces
strings—finds any printable strings in an object or binary file
stty—sets the options for a terminal
su—become superuser or another user
sum—calculates a checksum for a file
sync—updates the superblock and sends changed blocks to disk
tabs—set tab stops on a terminal
tail—displays the tail end of a file.
talk—allows you to talk to another user
tar—stores and retrieves files from an archive file, normally a tape device
tee—replicates the standard output
telnet—communicates with a remote host
test—evaluates an expression
time—displays a summary of time used by this shell and its children
timex—times a command; reports process data and system activity
touch—updates access time and/or modification time of a file
tput—initializes a terminal or queries the terminfo database
tr—translates characters
true—provide successful exit status
tsort —topological sort
tty—gets the name of the terminal
umask—sets file-creation mode mask for permissions
uname—prints name of current machine
uncompress—restores files to their original state after they have been compressed using the
compress command
uniq—reports on duplicate lines in a file
units—converts quantities expressed in standard scales to other scales
unpack—expands files created by pack
uucp—copy files to another system, UNIX-to-UNIX system copy
uuencode—uuencode, uudecode—encode a binary file into ASCII text in order to send it through email,or convert it back into its original form
wc—counts lines, words, and characters
what—extracts SCCS version information from a file by printing information found after the @(#)
pattern
which—locates a command and displays its pathname or alias (UCB)
whereis—locates the binary, source, and manual page files for a command (UCB)
who—displays who is logged on the system
write—writes a message to another user
xargs—constructs an argument list(s) and executes a command
zcat—uncompress a compressed file to standard output. Same as uncompress –c
at—at, batch—execute commands at a later time
awk—pattern scanning and processing language
banner—make posters
basename—with a directory name delivers portions of the pathname
bc—processes precision arithmetic
bdiff—compares two big files
cal—displays a calendar
cat—concatenates and displays files
chmod—change the permissions mode of a file
chown—changes owner of file
clear—clears the terminal screen
cmp—compares two files
compress—compress, uncompress, zcat compress, uncompress files, or display expanded files
cp—copies files
cpio—copy file archives in and out
cron—the clock daemon
crypt—encodes or decodes a file
cut—removes selected fields or characters from each line of a file
date—displays the date and time or sets the date
diff—compares two files for differences diff [–bitw] [–c | –Cn
du—summarizes disk usage
echo—echoes arguments
egrep—searches a file for a pattern using full regular expressions
expr—evaluates arguments as an expression
fgrep—search a file for a character string
file—determines the type of a file by looking at its contents
find—finds files
finger—displays information about local and remote users
fmt—simple text formatters
fold—folds long lines
ftp—file transfer program
getopt(s)—parses command line options
grep—searches a file for a pattern
groups—prints group membership of user
id—prints the username, user ID, group name and group ID
jsh—the standard, job control shell
line—reads one line
logname—gets the name of the user running the process
lp—sends output to a printer (AT&T)
lpr—sends output to a printer (UCB)
lpstat—print information about the status of the LP print service (AT&T)
lpq—print information about the status of the printer (UCB)
ls—lists contents of directory
mail—mail, rmail—read mail or send mail to users
mailx—interactive message processing system
make—maintains, updates, and regenerates groups of related programs and files
mesg—permits or denies messages resulting from the write command
mkdir—creates a directory
more—browse or page through a text file
mv—move or rename files
nawk—pattern scanning and processing language
newgrp—log in to a new group
news—prints news items
nice—runs a command at low priority
nohup—makes commands immune to hangups and quits
od—octal dump
pack—pack, pcat, unpack—compresses and expands files
passwd—changes the login password and password attributes
paste—merges same lines of several files or subsequent lines of one file
pcat—(see pack)
pg—displays files a page at a time
pr—prints files
ps—reports process status
pwd—displays the present working directory name
rcp—remote file copy
rlogin—remote login
rm—removes files from directories
rmdir—removes a directory
rsh—starts a remote shell
ruptime—shows the host status of local machines
rwho—who is logged in on local machines
script—creates a typescript of a terminal session
sed—streamlined editor
size—prints section sizes in bytes of object files
sleep—suspends execution for some number of seconds
sort—sort and/or merge files
spell—finds spelling errors
split—splits a file into pieces
strings—finds any printable strings in an object or binary file
stty—sets the options for a terminal
su—become superuser or another user
sum—calculates a checksum for a file
sync—updates the superblock and sends changed blocks to disk
tabs—set tab stops on a terminal
tail—displays the tail end of a file.
talk—allows you to talk to another user
tar—stores and retrieves files from an archive file, normally a tape device
tee—replicates the standard output
telnet—communicates with a remote host
test—evaluates an expression
time—displays a summary of time used by this shell and its children
timex—times a command; reports process data and system activity
touch—updates access time and/or modification time of a file
tput—initializes a terminal or queries the terminfo database
tr—translates characters
true—provide successful exit status
tsort —topological sort
tty—gets the name of the terminal
umask—sets file-creation mode mask for permissions
uname—prints name of current machine
uncompress—restores files to their original state after they have been compressed using the
compress command
uniq—reports on duplicate lines in a file
units—converts quantities expressed in standard scales to other scales
unpack—expands files created by pack
uucp—copy files to another system, UNIX-to-UNIX system copy
uuencode—uuencode, uudecode—encode a binary file into ASCII text in order to send it through email,or convert it back into its original form
wc—counts lines, words, and characters
what—extracts SCCS version information from a file by printing information found after the @(#)
pattern
which—locates a command and displays its pathname or alias (UCB)
whereis—locates the binary, source, and manual page files for a command (UCB)
who—displays who is logged on the system
write—writes a message to another user
xargs—constructs an argument list(s) and executes a command
zcat—uncompress a compressed file to standard output. Same as uncompress –c
const char *p and char * const p
Difference between const char *p and char * const p?
The former declares a pointer to a constant character; the latter declares a constant pointer to a character.
"char const *p" is a pointer to a constant character (you can't change the character);
"char * const p" is a constant pointer to a (variable) character (i.e. you can't change the pointer).
const char *p => cannot change the contents to which it is pointing
char *const p => cannot change the pointer address..
const char *p => cannot change the contents to which it is pointing
char *const p => cannot change the pointer address to which it is pointing
The former declares a pointer to a constant character; the latter declares a constant pointer to a character.
"char const *p" is a pointer to a constant character (you can't change the character);
"char * const p" is a constant pointer to a (variable) character (i.e. you can't change the pointer).
const char *p => cannot change the contents to which it is pointing
char *const p => cannot change the pointer address..
const char *p => cannot change the contents to which it is pointing
char *const p => cannot change the pointer address to which it is pointing
A shallow copy , Deep Copy and Smart Pointer
A shallow copy of an object copies all of the member field values. This works well if the fields are values, but may not be what you want for fields that point to dynamically allocated memory. The pointer will be copied. but the memory it points to will not be copied -- the field in both the original object and the copy will then point to the same dynamically allocated memory, which is not usually what you want. The default copy constructor and assignment operator make shallow copies.
A deep copy copies all fields, and makes copies of dynamically allocated memory pointed to by the fields. To make a deep copy, you must write a copy constructor and overload the assignment operator, otherwise the copy will point to the original, with disasterous consequences.
Smart Pointer
The simplest example of a smart pointer is auto_ptr, which is included in the standard C++ library. You can find it in the header.
Automatic cleanup don't need to remember to free the pointer,
Automatic initialization don't need to initialize the auto_ptr to NULL
Dangling pointers a pointer that points to an object that is already deleted
Since C++ does not provide automatic garbage collection like some other languages, smart pointers can be used for that purpose.
A deep copy copies all fields, and makes copies of dynamically allocated memory pointed to by the fields. To make a deep copy, you must write a copy constructor and overload the assignment operator, otherwise the copy will point to the original, with disasterous consequences.
Smart Pointer
The simplest example of a smart pointer is auto_ptr, which is included in the standard C++ library. You can find it in the header
Automatic cleanup don't need to remember to free the pointer,
Automatic initialization don't need to initialize the auto_ptr to NULL
Dangling pointers a pointer that points to an object that is already deleted
Since C++ does not provide automatic garbage collection like some other languages, smart pointers can be used for that purpose.
AVOID CURSORS
If possible, avoid using SQL Server cursors. They generally use a lot of SQL Server resources and reduce the performance and scalability of your applications. If you need to perform row-by-row operations, try to find another method to perform the task.
Here are some alternatives to using a cursor:
• Use WHILE LOOPS
• Use temp tables
• Use derived tables
• Use correlated sub-queries
• Use the CASE statement
• Perform multiple queries
If you do find you must use a cursor, try to reduce the number of records to process.
One way to do this is to move the records that need to be processed into a temp table first, and then create the cursor to use the records in the temp table, not from the original table. This of course assumes that the subsets of records to be inserted into the temp table are substantially less than those in the original table.
The lower the number of records to process, the faster the cursor will finish.
If you have no choice but to use a server-side cursor in your application, try to use a FORWARD-ONLY or FAST-FORWARD, READ-ONLY cursor. When working with unidirectional, read-only data, use the FAST_FORWARD option instead of the FORWARD_ONLY option, as it has some internal performance optimizations to speed performance. This type of cursor produces the least amount of overhead on SQL Server.
If you are unable to use a fast-forward cursor, then try the following cursors in this order, until you find one that meets your needs. They are listed in the order of their performance characteristics, from fastest to slowest: dynamic, static, and keyset.
Avoid using static/insensitive and keyset cursors, unless you have no other choice. This is because they cause a temporary table to be created in TEMPDB, which increases overhead and can cause resource contention issues.
If you have no choice but to use cursors in your application, try to locate the SQL Server tempdb database on its own physical device for best performance. This is because cursors use the tempdb for temporary storage of cursor data. The faster your disk array running tempdb, the faster your cursor will be.
Using cursors can reduce concurrency and lead to unnecessary locking and blocking. To help avoid this, use the READ_ONLY cursor option if applicable, or if you need to perform updates, try to use the OPTIMISTIC cursor option to reduce locking. Try to avoid the SCROLL_LOCKS cursor option, which reduces concurrency.
When you are done using a cursor, don't just CLOSE it, you must also DEALLOCATE it. Deallocation is required to free up the SQL Server resources used by the cursor. If you only CLOSE the cursor, locks are freed, but SQL Server resources are not. If you don't DEALLOCATE your cursors, the resources used by the cursor will stay allocated, degrading the performance of your server until they are released.
If it is appropriate for your application, try to load the cursor as soon as possible by moving to the last row of the result set. This releases the share locks created when the cursor was built, freeing up SQL Server resources.
If you have to use a cursor because your application needs to manually scroll through records and update them, try to avoid client-side cursors, unless the number of rows is small or the data is static. If the number of rows is large, or the data is not static, consider using a server-side keyset cursor instead of a client-side cursor. Performance is usually boosted because of a reduction in network traffic between the client and the server. For optimum performance, you may have to try both types of cursors under realistic loads to determine which is best for your particular environment.
When using a server-side cursor, always try to fetch as small a result set as possible. This includes fetching only those rows and columns the client needs immediately. The smaller the cursor, no matter what type of server-side cursor it is, the fewer resources it will use, and performance will benefit.
If you need to perform a JOIN as part of your cursor, keyset and static cursors are generally faster than dynamic cursors, and should be used when possible.
Here are some alternatives to using a cursor:
• Use WHILE LOOPS
• Use temp tables
• Use derived tables
• Use correlated sub-queries
• Use the CASE statement
• Perform multiple queries
If you do find you must use a cursor, try to reduce the number of records to process.
One way to do this is to move the records that need to be processed into a temp table first, and then create the cursor to use the records in the temp table, not from the original table. This of course assumes that the subsets of records to be inserted into the temp table are substantially less than those in the original table.
The lower the number of records to process, the faster the cursor will finish.
If you have no choice but to use a server-side cursor in your application, try to use a FORWARD-ONLY or FAST-FORWARD, READ-ONLY cursor. When working with unidirectional, read-only data, use the FAST_FORWARD option instead of the FORWARD_ONLY option, as it has some internal performance optimizations to speed performance. This type of cursor produces the least amount of overhead on SQL Server.
If you are unable to use a fast-forward cursor, then try the following cursors in this order, until you find one that meets your needs. They are listed in the order of their performance characteristics, from fastest to slowest: dynamic, static, and keyset.
Avoid using static/insensitive and keyset cursors, unless you have no other choice. This is because they cause a temporary table to be created in TEMPDB, which increases overhead and can cause resource contention issues.
If you have no choice but to use cursors in your application, try to locate the SQL Server tempdb database on its own physical device for best performance. This is because cursors use the tempdb for temporary storage of cursor data. The faster your disk array running tempdb, the faster your cursor will be.
Using cursors can reduce concurrency and lead to unnecessary locking and blocking. To help avoid this, use the READ_ONLY cursor option if applicable, or if you need to perform updates, try to use the OPTIMISTIC cursor option to reduce locking. Try to avoid the SCROLL_LOCKS cursor option, which reduces concurrency.
When you are done using a cursor, don't just CLOSE it, you must also DEALLOCATE it. Deallocation is required to free up the SQL Server resources used by the cursor. If you only CLOSE the cursor, locks are freed, but SQL Server resources are not. If you don't DEALLOCATE your cursors, the resources used by the cursor will stay allocated, degrading the performance of your server until they are released.
If it is appropriate for your application, try to load the cursor as soon as possible by moving to the last row of the result set. This releases the share locks created when the cursor was built, freeing up SQL Server resources.
If you have to use a cursor because your application needs to manually scroll through records and update them, try to avoid client-side cursors, unless the number of rows is small or the data is static. If the number of rows is large, or the data is not static, consider using a server-side keyset cursor instead of a client-side cursor. Performance is usually boosted because of a reduction in network traffic between the client and the server. For optimum performance, you may have to try both types of cursors under realistic loads to determine which is best for your particular environment.
When using a server-side cursor, always try to fetch as small a result set as possible. This includes fetching only those rows and columns the client needs immediately. The smaller the cursor, no matter what type of server-side cursor it is, the fewer resources it will use, and performance will benefit.
If you need to perform a JOIN as part of your cursor, keyset and static cursors are generally faster than dynamic cursors, and should be used when possible.
Subscribe to:
Posts (Atom)