Defragmenting my drives manually is one of those things like closing apps on my phone I am not using. Everyone says there is no need to do it. It might even be slightly worse overall. I even have enough self control to avoid doing it and have for years...
But deep down in my heart, I truly FEEL like if I did it would improve things... somehow.
I feel so seen right now. My partner complains about me closing apps on my phone consistently. I also have to hold myself back from defragging and continually cleaning up my storage drives.
I too am nearly certain that the positive benefits are approaching nil, but I still feel it should be helpful.
At least on iPhone, it basically doesn't do anything.
Apple aggressively kills all background apps automatically. the "open" apps in the switcher are almost always just screenshots of previously-open-but-now-closed apps. if it was recent, the latest app or two in the switcher might actually be open, but rarely more than that.
Despite this, many of my relatives have somehow learned this habit of opening the switcher and closing all apps when they are done with them.
Android has always been quite vigilant and is prompt to kill off any app at the MFA login screen while I am consulting the Oracle of Authenticator in another task. Thanks Android!
That being said, I dislike having 3 dozen apps open because I simply can’t quickly find and switch to the ones I’m using. So when I enter a new situation, I swipe all the way left, and Clear All apps, to start over with a tabula rasa.
The app I use is gone from the app store, but like Spotlight or the Windows Start Menu > type-to-search (before it got utterly shittified with web search/does it have AI now?), I have an Android app called App Search+ on the bottom left of my Android home screen.
Open it, and it opens a list of recently searched for apps. The focus is on the top search bar (and the keyboard has also popped up), and I can type to find any app I have installed.
iOS has never worked the way you are describing. It's easy to disprove your claim that the app is killed by just switching between different apps you have open. iOS keeps the apps open to make it fast to switch back to them.
The document is clear that the expectation from the developer is that you need to be ready for your app to be snapshotted and frozen/killed at any moment when it is not on the foreground.
Well, it closes the apps somewhat arbitrarily, but they definitely arent all open. If the most recent app is a resource-intensive game or something it will more aggressively close things. but you can have more open if they are all lightweight apps.
But the switcher shows every app ever opened and tries to pretend these apps aren't being closed.
Right now for instance, i just checked and as best i can tell, the last 3 were open. going back any further and i could tell the app had to reload. but i can scroll to the left back forever. the one on the furthest left i easily haven't opened for months
Being able to go back 3 already contradicts the claim that it was rare to be more than 2. It works the same as Android. Android kills apps based off of resource usage but keeps screenshots of them in the task switcher.
After digging through a recently passed relative’s hard drive (the computer barely functioned so I pulled it) and having to search through years of garbage that where never deleted to find the photos, Word documents, and WordPerfect documents my aunt ‘knew’ where stored there somewhere, this is not an insignificant gift to leave my family with.
I discovered that tested performance on copper-rs a high performance OS for robotics when I run it on ext4 and what is annoying in robotics is the max latency & jitter. Especially that we allocate large slabs in copper so this is not helping at all.
This is on a slow device but it might be worse on a fast one as the extra allocation of
small extents start to hit harder on the host side:
(fragmented relative to baseline)
bandwidth: 100.65 -> 94.74 MiB/s (-5.86%)
elapsed time: 339.550 -> 360.699 s (+6.23%)
mean latency: 9935.7 -> 10554.6 us (+6.23%)
p50 latency: 6715.2 -> 6838.9 us (+1.84%)
p95 latency: 19394.4 -> 20599.9 us (+6.22%)
p99 latency: 131498.2 -> 131988.3 us (+0.37%)
max latency: 183713.2 -> 624392.9 us (+239.87%)
jitter stddev: 17823.1 -> 19806.0 us (+11.13%)
jitter CV: 179.38 -> 187.65% (+4.61%)
Also check your nvme granularity with fstrim -D if you are on 4KB your nvme is so fine grain that it doesn't matter much but if it is 64KB like my main one. Ouch, those small files in the middle of the 64KB won't magically go away.
In case you (or others aren’t aware) the FRAG value ZFS shows, is an arbitrary calculation to help give a value to free space fragmentation.
The purpose is to have a sense of when ZFS may developing trouble quickly finding contiguous blocks of free space to dump new data into. Mostly only a concern for high active and highly full pools. When you start watching the number climb past double digits, you may start finding performance issues but it’s a very contextual thing, rather than “at x value it’s bad”.
Basically it has nothing to do with written file fragmentation, or potential read speed.
Because ZFS is a copy on write filsystem, attempts at defragmentation an active pool are generally not effective. An exception would be a pool that is full of large files that was written in a non contiguous way (like torrents), in which case copying the data to a new dataset and deleting the old roughly achieves the purpose. Or you could use something like https://github.com/salesforce/zfs_defrag
If it makes it feel faster, it makes it feel faster.
It's like those copper bangles with magnets that people swear makes their joints less creaky.
My example is a daith piercing, that's when you pierce a ring through a fold of cartilage in a certain spot inside your ear. It's supposed to stop you getting migraines, but there's no sensible mechanism for this to work. It's all woo and bunkum, apparently.
But I've had four migraines in eight years as opposed to four every month.
Still, given SSDs have a lifespan and given few of my files are that large, I think I'm personally okay without defragmenting. Those numbers are impressive but, most of 10s of thousands of files are source code text files. Asking online what the real-world loss is from not defragmenting
You are giving up virtually 0% to 3% of real-world performance. In daily development, media editing, and standard system use, defragmenting your SSD will yield no human-noticeable speedup.
You cannot directly inspect the degree of fragmentation, because it has less to do with being contiguous in the Logical Block Address (LBA) space and more to do with having been written at the same time. To properly defragment a file on a SSD, you pretty much need to sequentially re-write the whole file in one go, to a newly-allocated part of the drive's LBA space.
to a newly-allocated part of the drive's LBA space
Which is rather a case for home use, not practical for highly parallel access industrial use cases. That's why the issue became a research topic. For the first time I saw a proof that the "fact" SSDs do not need defragmentation is actually a myth. Well, you still want to avoid explicit defragmentation (waste of time and SSD lifetime) by filesystem driver submitting additional hints (using new NVMe extensions) to the SSD controller about what blocks belong to the same file, so that SSD can place them for an optimal sequential access with properly interleaving (not necessary strictly physically consecutively whatever this means on SSD).
Otherwise the claimed defragmentation here is not actually resulting in sequential data.
I haven't measured it but the SSD firmware should be able to look up where the next block is speculatively in order to be able to immediately start sending it if the host tries and read the next block (as opposed to a random one).
Comments
Beautiful.
Defragmenting my drives manually is one of those things like closing apps on my phone I am not using. Everyone says there is no need to do it. It might even be slightly worse overall. I even have enough self control to avoid doing it and have for years...
But deep down in my heart, I truly FEEL like if I did it would improve things... somehow.
I feel so seen right now. My partner complains about me closing apps on my phone consistently. I also have to hold myself back from defragging and continually cleaning up my storage drives.
I too am nearly certain that the positive benefits are approaching nil, but I still feel it should be helpful.
I'd be curious to know the reqson why she would care on her own phone and why it would be an issue on someone else's phone?
At least on iPhone, it basically doesn't do anything. Apple aggressively kills all background apps automatically. the "open" apps in the switcher are almost always just screenshots of previously-open-but-now-closed apps. if it was recent, the latest app or two in the switcher might actually be open, but rarely more than that.
Despite this, many of my relatives have somehow learned this habit of opening the switcher and closing all apps when they are done with them.
Android has always been quite vigilant and is prompt to kill off any app at the MFA login screen while I am consulting the Oracle of Authenticator in another task. Thanks Android!
That being said, I dislike having 3 dozen apps open because I simply can’t quickly find and switch to the ones I’m using. So when I enter a new situation, I swipe all the way left, and Clear All apps, to start over with a tabula rasa.
The app I use is gone from the app store, but like Spotlight or the Windows Start Menu > type-to-search (before it got utterly shittified with web search/does it have AI now?), I have an Android app called App Search+ on the bottom left of my Android home screen.
Open it, and it opens a list of recently searched for apps. The focus is on the top search bar (and the keyboard has also popped up), and I can type to find any app I have installed.
i’m convinced the real reason people do this is because they don’t want anyone to see what apps they were using.
If you just have every app in the switcher, it becomes useless too.
iOS has never worked the way you are describing. It's easy to disprove your claim that the app is killed by just switching between different apps you have open. iOS keeps the apps open to make it fast to switch back to them.
This is literally always how iOS has worked. You can refer to the application lifecycle documentation: https://developer.apple.com/documentation/uikit/managing-you...
What point are you trying to make. The documentation doesn't say the OS only keeps a single app process running at a time.
The document is clear that the expectation from the developer is that you need to be ready for your app to be snapshotted and frozen/killed at any moment when it is not on the foreground.
The page you linked to doesn't say that anywhere.
Well, it closes the apps somewhat arbitrarily, but they definitely arent all open. If the most recent app is a resource-intensive game or something it will more aggressively close things. but you can have more open if they are all lightweight apps.
But the switcher shows every app ever opened and tries to pretend these apps aren't being closed. Right now for instance, i just checked and as best i can tell, the last 3 were open. going back any further and i could tell the app had to reload. but i can scroll to the left back forever. the one on the furthest left i easily haven't opened for months
I do this also - but wonder because I'll have some app open, phone gets warm. Close the app and it cools down.
Being able to go back 3 already contradicts the claim that it was rare to be more than 2. It works the same as Android. Android kills apps based off of resource usage but keeps screenshots of them in the task switcher.
When you die, you will leave behind nice orderly contiguous files. It's thoughtful. hehe
After digging through a recently passed relative’s hard drive (the computer barely functioned so I pulled it) and having to search through years of garbage that where never deleted to find the photos, Word documents, and WordPerfect documents my aunt ‘knew’ where stored there somewhere, this is not an insignificant gift to leave my family with.
I discovered that tested performance on copper-rs a high performance OS for robotics when I run it on ext4 and what is annoying in robotics is the max latency & jitter. Especially that we allocate large slabs in copper so this is not helping at all.
This is on a slow device but it might be worse on a fast one as the extra allocation of small extents start to hit harder on the host side:
(fragmented relative to baseline) bandwidth: 100.65 -> 94.74 MiB/s (-5.86%) elapsed time: 339.550 -> 360.699 s (+6.23%) mean latency: 9935.7 -> 10554.6 us (+6.23%) p50 latency: 6715.2 -> 6838.9 us (+1.84%) p95 latency: 19394.4 -> 20599.9 us (+6.22%) p99 latency: 131498.2 -> 131988.3 us (+0.37%) max latency: 183713.2 -> 624392.9 us (+239.87%) jitter stddev: 17823.1 -> 19806.0 us (+11.13%) jitter CV: 179.38 -> 187.65% (+4.61%)
Also check your nvme granularity with fstrim -D if you are on 4KB your nvme is so fine grain that it doesn't matter much but if it is 64KB like my main one. Ouch, those small files in the middle of the 64KB won't magically go away.
only HDD drives I use are in a ZFS pool which has a very usable fragmentation value from zpool.
In case you (or others aren’t aware) the FRAG value ZFS shows, is an arbitrary calculation to help give a value to free space fragmentation.
The purpose is to have a sense of when ZFS may developing trouble quickly finding contiguous blocks of free space to dump new data into. Mostly only a concern for high active and highly full pools. When you start watching the number climb past double digits, you may start finding performance issues but it’s a very contextual thing, rather than “at x value it’s bad”.
Basically it has nothing to do with written file fragmentation, or potential read speed.
Because ZFS is a copy on write filsystem, attempts at defragmentation an active pool are generally not effective. An exception would be a pool that is full of large files that was written in a non contiguous way (like torrents), in which case copying the data to a new dataset and deleting the old roughly achieves the purpose. Or you could use something like https://github.com/salesforce/zfs_defrag
Wow! I did not know that, thank you for the info.
Indeed torrents were the most obvious cause of the fragmentation I saw.
If it makes it feel faster, it makes it feel faster.
It's like those copper bangles with magnets that people swear makes their joints less creaky.
My example is a daith piercing, that's when you pierce a ring through a fold of cartilage in a certain spot inside your ear. It's supposed to stop you getting migraines, but there's no sensible mechanism for this to work. It's all woo and bunkum, apparently.
But I've had four migraines in eight years as opposed to four every month.
So, if it feels faster, it might well be.
Except for some backups, all of my machines use solid state storage so I don't think there is a point for most people, is there?
Sequential reads are faster for SSDs. As long as defragmentation is reducing the chance of doing a random read, it is beneficial.
OH! TIL! Apparently
Still, given SSDs have a lifespan and given few of my files are that large, I think I'm personally okay without defragmenting. Those numbers are impressive but, most of 10s of thousands of files are source code text files. Asking online what the real-world loss is from not defragmentingIs it possible to determine which sectors are physically sequential given remapping for wear-leveling?
Otherwise the claimed defragmentation here is not actually resulting in sequential data.
You cannot directly inspect the degree of fragmentation, because it has less to do with being contiguous in the Logical Block Address (LBA) space and more to do with having been written at the same time. To properly defragment a file on a SSD, you pretty much need to sequentially re-write the whole file in one go, to a newly-allocated part of the drive's LBA space.
Which is rather a case for home use, not practical for highly parallel access industrial use cases. That's why the issue became a research topic. For the first time I saw a proof that the "fact" SSDs do not need defragmentation is actually a myth. Well, you still want to avoid explicit defragmentation (waste of time and SSD lifetime) by filesystem driver submitting additional hints (using new NVMe extensions) to the SSD controller about what blocks belong to the same file, so that SSD can place them for an optimal sequential access with properly interleaving (not necessary strictly physically consecutively whatever this means on SSD).
https://www.usenix.org/conference/fast24/presentation/jun
I haven't measured it but the SSD firmware should be able to look up where the next block is speculatively in order to be able to immediately start sending it if the host tries and read the next block (as opposed to a random one).
Prefetching applies to SSDs too, and that's generally sequential in some sense.