Why is there unpartitioned space on my SD card?

Question

First off, I'm running Rasbian with NOOBS.

I was following a tutorial on creating a new partition here: How do I create and mount a partition using the remainder of my SD card? And I started wondering why there's un-partitioned space on the sd card to begin with? Can I count on this always to be the case if I install Raspbian using NOOBS? Or should it be expected that the root partition takes up the entire remaining space? Thanks in advance.

Info for accepted answer

The reason I asked this question is because I'm updating a fleet of raspberry pi's remotely. Part of my update requires making a new partition on the disk. I am worried that the method I cited won't work for every SD card on my devices even though they are cloned from the same image. It worked for the SD card I'm testing on but how do I know there will be unpartitioned space on EVERY card?

The answer below tells me basically... it's not guaranteed and also resizing the root file system is impossible without unmounting it I believe. So basically I can either first see if there is unpartitioned space and apply the update, or get notified that a certain device can't be updated... shoot

Edit #2:

@crasic pointed out that it is possible to resize the root partition via the boot parameters in /boot/config.txt. See his link in his answer

Answer 1

By default, on first boot, for recent versions, Raspbian image expands the partition to fill the entire space. You can always trigger this resize manually as well using raspi-config as was indicated in an answer to the question you cited.

However, NOOBS changes this behavior by doing its own partitioning and perhaps not invoking this resize script. There is also an option to reserve a certain amount as a data partition directly from NOOBS. I do not believe it is default to leave unpartitioned space when doing a flat install with NOOBS.

If you need this space to always exist, you can either explicitly install with reserved space, manually partition, or modify the resize script to leave some room.

For details on how the resize-on-first-boot works, see my other answer.

There is space left because the OS image is shrunk to a minimal size to reduce the payload size and to support a wide range of SD card sizes (Otherwise you would be forced to use images for specific SD Card Sizes). The operating system provides tools to resize the partition, and has a boot script to automate the process.