Archived Forum

Is the 10Mb buffer sufficient? with the variations in SD cards, should this be increased/decreased?

10MB might be cutting it a bit close. I'm thinking 20, or even 30MB might be "cheap insurance," given the low cost of SD cards.

Thanks Bill, I'm mindful that space is at a premium on the 4Gb cards, as there is only 300Mb for the data partition. Do you think that the buffer size should increase proportional to the card size?

I took a look at two 4 GB microSD cards, one made by Transcend, the other by Adata.

The difference surprised me, as it was only 3MB. So perhaps a 10MB buffer is sufficient for a 4GB card.

Here's a comparison between two 8GB microSD cards:

SanDisk 7,948,206,080 bytes

Toshiba 7,746,879,488 bytes

Good call with regards to increasing the buffer size as the card size increases.

Isn't that about 200Mb difference?

I'm seeing 7,746,878,976 bytes in my cheepo Ebay card, which is not dissimilar to the Toshiba card.

Paul

Isn't that about 200Mb difference?

Almost. (a tad less actually, but not much). Hence my comment about your good call...

Good idea re this script. Creating partitions correctly is a pain! I've added note this thread from the read-only setup guide 

https://github.com/emoncms/emoncms/commit/831d9f4296548bbd301e094a1c0369...

Glyn, almost there with the script, it works fine, but concerned about the previous 2 posts. Do we really need a 200Mb buffer on 8Gb cards? Seems out of proportion.... 

Paul

Since the script must be run with root privileges, the sudo commands inside the script aren't needed.

Thanks Bill, there's still a few minor changes needed before its finally finished.

If anyone wants to check & report their actual SD card size, the command is:
$ sudo fdisk -l
The card size in bytes should be on the first line of info.

Paul

A Sandisk microSD HC: 7951351808 bytes.

(and fwiw, an otherwise the same 4 GB: 4004511744 bytes)

The 'sdpart' script has now been added to the 'usefulscripts' repo if anyone wishes to use it.

For now, the script creates a 300Mb data partition plus a 10Mb buffer for 4Gb cards, and 1Gb data partition plus 50Mb buffer for larger cards.

I'm not sure if the 50Mb buffer is ideal having read the above posts, so we may need to revisit this at a later time. But at least it's 50Mb more than if the root partition was expanded by using raspi-config.

Just to clarify, the easiest way to use the script is to install the operating system on the SD card, and run the script INSTEAD of using the system configuration tool - raspi-config to expand the filesystem.

The script can be downloaded and run by;

cd && git clone https://github.com/emoncms/usefulscripts.git
sudo usefulscripts/sdpart/./sdpart

​Paul

Hi Paul.

Replying to your question on the emonSD12Feb16 thread.

I used the sdpart script on the previous stable version with no issues. Straight forward run and reboot. All OK.

On emonSD12Feb16 I see a few "Value out of range" errors. Don't know what they mean so attached the output messages when script is run.

Thanks for the feedback.

It would appear from your log there were 3 partitions on the SD card already before running the script;

/dev/mmcblk0p1    8192  131071  122880    60M  c W95 FAT32 (LBA)
/dev/mmcblk0p2  131072 7391231 7260160   3.5G 83 Linux
/dev/mmcblk0p3 7391232 7801780  410549 200.5M 83 Linux

Whereas a default Jessie/Wheezy installation should just create two partitions.

The script is intended to be used by users building their systems using the install guides, not using the image files, which I assume you did?

Paul

Hi Paul, 

How difficult would it be to extend this script, or write another, to enable users of the pre-built image to extend the data partition to fill the SD card. 

Default emonPi image has a RW ~/data partition of 150MB. If using an SD card > 4GB (default emonPi is 8GB) the data partition can be expanded to fill the rest of the SD

I'll take a look and see what I can do.
Give me a few days, I'm not at home at the moment.

Paul

No rush, it was just an idea. It's an issue when a user attempts to restore a large backup via the new backup module:

https://github.com/emoncms/backup/

I've just completed the script to use in conjunction with the emoncms imagefile, and a git pull request has been submitted. (anyone wanting to try it now can git clone my usefulscripts repo)

The script - sdpart_imagefile will hopefully be merged into the 'usefulscripts' repo, and will become part of the imagefile itself, so after writing the emoncms imagefile to (any size) SD Card, it will be run by the following command;

sudo /home/pi/usefulscripts/sdpart/./sdpart_imagefile

For safety, the script will only run if you are using an emoncms imagefile, and it will analyze your SD card, and determine a safe maximum size to expand the /data partition, allowing a buffer at the end of the disk.

Using fdisk, the script will then resize the partition as above, and automatically reboot your system so the changes can take effect.

Following a reboot, the new extended partition will be checked by e2fsck for errors, before using resize2fs to extend the filesystem to fill the partition. This stage takes a long time, depending upon the SD card size, but for example a 32Gb SD card takes about 20 minutes to complete, so it's important not to disturb the script's operation.

Once complete, the script will signify that it's ended by shutting down the pi (poweroff), so at that stage, you can re-apply power and it should be good to go.

The short version of the above is, run the script, type 'Y' to proceed, come back about 20 minutes later.

Paul

Paul - Thank you for creating this!  As I wait for the emonSD-20Feb16 image to copy to a spare SD card I was reading through the new script.  In the comments I see a mention of "preferred size" for the data partition. How do I set the /data partition size? I am looking to expand the /data partition from 200 MB to about 500 MB on a 8 GB SD card.

The script will calculate the /data partition size to 'almost' fill all remaining space on the SD card, so on a 8Gb card it would probably be about 5Gb.

If you expand the /data partition to just 500Mb, have you plans to use the remaining 5Gb of unused disk space?

If you need bespoke partition sizes, I would recommend not using the script, and use something like Gparted to modify/create the partitions/filesystems, it's far more flexible.

The script was really intended for users wanting a quick & easy way to get up and running.

Paul

Paul - quick & easy are perfect!  I ran the script and it partitioned my SD card as expected (except I used a 16 GB card instead of an 8 GB).  There was one odd error I saw:

Device         Boot   Start      End  Sectors  Size Id Type
/dev/mmcblk0p1         8192   131071   122880   60M  c W95 FAT32 (LBA)
/dev/mmcblk0p2       131072  7391231  7260160  3.5G 83 Linux
/dev/mmcblk0p3      7391232 31188992 23797761 11.4G 83 Linux

Command (m for help): The partition table has been altered.
Calling ioctl() to re-read partition table.
Re-reading the partition table failed.: Device or resource busy

The kernel still uses the old table. The new table will be used at the next reboot
 or after you run partprobe(8) or kpartx(8).

Writing the resize2fs_once script

Pleased that it worked OK Jon

The error message can be disregarded, it's just really telling you that the system needs to be rebooted before the new partition table that has just been written can be read. The script at this point then does automatically reboot, and the new partition table is read by the system, enabling the filesystem to be extended.

If we had disregarded the message and not rebooted, we would have been unable to check and expand the filesystem, and errors would then be introduced.

A solution could be adding a line or two underneath the error message, reassuring users to disregard it. So instead of;

Writing the resize2fs_once script

It could read;

This error message can however be disregarded, because your system
is about to be rebooted, and the new partition table will then be read by your OS.

Would that help?

Paul

Edit - following contact from another user, the script has been updated to display the above. 

Can confirm that  sdpart_imagefile script works a treat for expanding the data partition to fill SD card. Tested with emonSD-20Feb16. Script pull request has now been merged into usefulscripts master branch. 

Hats of to Paul :-)