I tried:
sudo dd bs=4k if=/dev/mmcblk0 of=/media/1BAB47551C66A42B/raspbian_migs2.gz
It creates an .img file with 7.6 GB (card size, BUT what's on the card has 700 MB).
And:
sudo dd bs=4k if=/dev/mmcblk0 | gzip > /media/1BAB47551C66A42B/raspbian_migs2.gz
creates a .gz file with 2.7 GB.
The original Raspbian (Debian 7 (Wheezy)) from http://www.raspberrypi.org/downloads has 494.44 MiB.
From what's on the SD card, how can I make an image similar in size?
(I'm on Ubuntu.)
You mention in a comment to RooTer that A) you have reduced the initial partition size with gparted, but dd still copies the whole card, and B) that you want to include both partitions in the image.
Issue "A" is easy to explain: you are still copying the whole card because that's what /dev/mmcblk0 refers to. The individual partitions are of course /dev/mmcblk0p1 and /dev/mmcblk0p2. This is the complication in issue
"B", but you cannot simply dd each partition and concatenate the two files together, because of the partition table at the beginning of /dev/mmcblk0 which indexes the beginning and length of each partition. Without that, the image will be unusable.
However, you can get the length of each partition from fdisk -l, and use that to determine some parameters for dd. For example:
> fdisk -l /dev/mmcblk0
Disk /dev/mmcblk0: 16.1 GB, 16138633216 bytes
4 heads, 16 sectors/track, 492512 cylinders, total 31520768 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00017b69
Device Boot Start End Blocks Id System
/dev/mmcblk0p1 8192 122879 57344 c W95 FAT32 (LBA)
/dev/mmcblk0p2 122880 26746879 13312000 83 Linux
The "Start" and "End" units are sectors, and notice the sector size is given, 512 bytes. For /dev/mmcblk0p2, 26746879 (the last sector) - 122880 (the first sector) = 26623999 / 2 (for 2 sectors per kB) / 1024 (kB per MB) / 1024 (MB per GB) = 12.69, which I grew the partition using gparted to 12 GB, so this looks correct (really I should be using 1000 and not 1024 as the divisor with storage, which works out to 13.31 GB, but I suspect gparted and some other tools also use 1024).
So the first thing you want to check is that your second partition is really the smaller size that you set it to. Next, just use those numbers with dd; for me it would be:
dd if=/dev/mmcblk0 of=rpi.img bs=512 count=26746880
I've got an extra sector there to avoid any kind of off by one misunderstanding of how dd works. There's a simple way to check if this worked:
> fdisk -l rpi.img
Disk rpi.img: 102 MB, 102400000 bytes
255 heads, 63 sectors/track, 12 cylinders, total 200000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00017b69
Device Boot Start End Blocks Id System
rpi.img1 8192 122879 57344 c W95 FAT32 (LBA)
rpi.img2 122880 26746879 13312000 83 Linux
Notice there is a bit of a discrepancy here: the "Start" and "End" sectors match the original partition table, but the total size in the stats up top is only 102 MB! This is because I actually used count=200000 as the param to dd because I did not really want to bother with a 12 GB copy (notice also "total 200000 sectors"). The reason the table at the bottom does not reflect this is because fdisk is getting its information from the partition data copied verbatim at the beginning of the image from the beginning of the SD card, which, as I mentioned in the second paragraph, is vital to maintain. If I had (properly) copied the rest, the numbers would be copacetic and the image would be viable.
Give that a try. :)
I would guess problem lies in once used sectors that still have dirt in them. Once file is deleted, only metadata is removed from filesystem, not the data itself, thus leaving some random one-zeros instead of easy to compress zero-only blocks.
Easy solution, but requires rewriting all free space on the card. Remember that SD card lifespawn is limited by number of rewrites, so this is not preferred method.
dd bs=4M if=/dev/zero of=/root/junk
sync
rm junk
More involved solution, as you need to install zerofree on another computer which won't be using that SD card at the time.
zerofree /dev/mmcblk0p2
For more read up on http://intgat.tigress.co.uk/rmy/uml/index.html
What you have to remember is that doing dd of /dev/mmcblk0 you copy whole device, even if partitions are smaller. If you used raspi-config to expand main partition before doing one of above methods you will be fine.
P.S. If you don't mind changing format image file you can use partimage which for known filesystems omits freed blocks even if they still have some dirt in them. Again partimage is best to be used when filesystem is not mounted to avoid corruption of backup. You probably could get away with remounting it read-only, but I will leave it to your discretion.
Short answer -- use 2GB SD card.
Long answer, dd has no idea where the "good" data ends, you have to tell it somehow.
There are two ways, easiest is to use 2GB SD card, that will automatically stop copying beyond 2GB and will result in 500MB compressed file as you desire.
The other way, more complicated an involved one, is to calculate the correct data size from your partition table and specify this correct size as a parameters to dd command. You may use bs=XXX (block size) and count=XXX (block count) parameters for that purpose. For example, you may specify bs=10M for 10MB block size (that would definitely make copying much faster compared to 4k block size you use in your commands) and count=200 to copy 10MB * 200 = 2000MB (2GB). You may need to adjust block size and block count according to your SD card partition scheme.
dd - copy and convert is not the right tool to do the job you have requested. It is a low level, sector by sector copy (and convert) tool, which is great to copy boot sectors, format devices, and all sorts of low level tasks. When you use dd you are copying sector-by-sector to the image, even if it is not included in the File System Structure.
The images provided by the Raspberry Pi foundation, are specially compiled images with install scripts, unzipping binaries and initial setup, after which you have to get updates from the internet anyway - Which is all deliberate but quite a task to get it workin like this.
One popular solution to avoid copying empty sectors is to use a File Level copy system - and CloneZilla is standalone, bootable from a CD, similar to ye olde Norton Ghost but clonezilla supports the Linux (and more) file systems. So, it will only copy files that are in use and create a container from those files only. Reduces the size significantly!
I had this same exact question and wanted an easy to use tool. After searching and not finding one I wrote mkimg.sh. I outline the process I used at: https://raspberrypi.stackexchange.com/a/37899/32585
