The dd command without any options simply reads stdin and writes stdout; it's a copier like cat ("dd" stands for datadumper). If that were the end of it, there would be no reason for it to exist.
However, it can do much more, such as converting ASCII to EBCDIC and vice versa. The need for that has almost (but not quite) disappeared, yet dd still has use. It can block and unblock records (converting vast sequences of bytes to line oriented data and vice versa), programatically skip or seek past input or output (useful for reading or writing X bytes into a file), and more.
Given a file "t" containng "123456790",
dd if=t bs=1 skip=3
produces "4567890", and
echo "abc" | dd of=t seek=3 count=2 bs=1 conv=notrunc
leaves "t" as "123ab67890".
If "t" is instead:
123467890 123467890 123467890 123467890
Then "dd if=t conv=block cbs=10 " will produce exactly 40 bytes of "1234567890123456789012345678901234567890". Given those 40 bytes, "dd if=u conv=unblock cbs=5" will give you back 8 lines:
12345 67890 12345 67890 12345 67890 12345 67890
At the end of any transfer, dd always reports the number of full and partial blocks it read and wrote. Given our same file "t"
echo "1234567890" > t dd if=t ibs=3 obs=4 1234567890 3+1 records in 2+1 records out dd if=t ibs=3 obs=1 1234567890 3+1 records in 11+0 records out
Some databases work with fixed length records, and sometimes you need to convert those to something you can work with as ordinary text. The "dd" program is the answer for that. Another common need is to pad ascii text with white space to make it the lines a constant length (this might be needed to import into some other program or database). Perl or awk can handle that easily.
In this case, the request was to unblock every 756 bytes and create a constant length output:
First, to convert to lf every 756 characters:
dd if=yourfile of=newfile conv=unblock cbs=756
The next step is to add whitespace. Good old awk can always do that if it's text:
cat newfile | awk '{ printf "%-956s\n",$0 }'
Combine these together with
dd if=yourfile conv=unblock cbs=756 | awk '{ printf "%-956s\n",$0 }' >
Test with small files first to avoid typos.
dd can get a small file from your big file:
dd if=yourfile of=smallfile bs=756 count=3
"dd" can do all sorts of conversion for you - ebcdic to ascii, and more.
You can't use wild cards with "dd". However you can do something like this:
for i in VOL.* do echo "next disk" read ak dd if=$i of=/dev/fd0135ds18 bs=18k done
See xxd - a little known tool worth knowing
A customer used "dd" to store a file on tape and wondered why the file was different when he recovered it using dd.
The problem is that a tape is a block device - that is, it will always return a full block of data, a block being whatever its block size is. So unless the file written was an even multiple of the tapes block size, there will be nulls tacked on when you bring the file back from tape with dd.
Moral: use tar or cpio or something else that restores files, not blocks.
That's not to say dd is useless with tape. In fact a very common use is this:
tar cvf /tmp/mytar.tar dd if=/tmp/mytar.tar of=/dev/tape
This might be because you want to make multiple tapes of the same tar data; do the tar once, dd it as many times as you like. The resultant tapes can be extracted with tar.
Another very common use is across networks
tar cvf - . | rcmd otherbox dd of=/dev/tape
I couldn't tell you how many times I've done that.
You might also do this for performance - a different block size could write the tape more quickly. See Help with solving an error message on a cpio backup.
This thread has a an interesting deep dive into 'dd' and tape drive usage..
Got something to add? Send me email.
More Articles by Tony Lawrence © 2014-09-08 Tony Lawrence
C is quirky, flawed, and an enormous success. (Dennis Ritchie)
See also volcopy for a way to "dd" UNIX filesystems.
--BigDumbDinosaur
Mon Sep 8 17:06:13 2014: 12525 anonymous
This is fantastically useful even in 2014!
For smaller files, the GNU fold command can accomplish what dd and awk do. For example, "fold -b -w 150 file " will add a newline after every 150th byte
------------------------
Printer Friendly Version
dd Copyright © November 2003 Tony Lawrence
Have you tried Searching this site?
This is a Unix/Linux resource website. It contains technical articles about Unix, Linux and general computing related subjects, opinion, news, help files, how-to's, tutorials and more.
Contact us
Printer Friendly Version