I recently reimplemented some of the caches within RestKit using a `NSMutableDictionary` guarded by a dispatch queue instead of a `NSCache`/`NSMutableDictionary` + `NSRecursiveLock`. The general idea is that you use a concurrent dispatch queue to provide concurrent read access and then use barriers to obtain an exclusive write lock to the resource. There were significant performance benefits from the change as I was able to shift the cache updates on a miss into the background. The dispatch queue approach also outperformed `NSCache` significantly under the workloads I was testing -- it appears that some of its internal calculations on add/remove can be quite costly if you are adding/removing rapidly.
Its also very flexible, as you can do `dispatch_sync` if you need a synchronous fetch of the resource or use callback blocks to go as async as possible.
I've been thinking about this since yesterday, and it seems like a really good solution for some use cases, but isn't there quite a bit of overhead to using a dispatche queue for fine-grained operations? I mean it seems like the overhead of cross-thread communication would severely outweigh a simple synchronous lock call.
Then again, if you can do things like background non-essential operations, then the higher-level benefits can probably outweigh that.
Like I said, GCD has a heavy emphasis on performance. For example, a dispatch_sync will involve no cross-thread communication in the uncontended case, and the cost is comparable to taking a spinlock. A bit heavier, but not too much.
dispatch_async will necessarily be slower, but don't overestimate how much work is really going to happen in the cases where it matters.
Comments
Fascinating. I'll have to give that a shot. Thanks!
I recently reimplemented some of the caches within RestKit using a `NSMutableDictionary` guarded by a dispatch queue instead of a `NSCache`/`NSMutableDictionary` + `NSRecursiveLock`. The general idea is that you use a concurrent dispatch queue to provide concurrent read access and then use barriers to obtain an exclusive write lock to the resource. There were significant performance benefits from the change as I was able to shift the cache updates on a miss into the background. The dispatch queue approach also outperformed `NSCache` significantly under the workloads I was testing -- it appears that some of its internal calculations on add/remove can be quite costly if you are adding/removing rapidly.
Its also very flexible, as you can do `dispatch_sync` if you need a synchronous fetch of the resource or use callback blocks to go as async as possible.
I've been thinking about this since yesterday, and it seems like a really good solution for some use cases, but isn't there quite a bit of overhead to using a dispatche queue for fine-grained operations? I mean it seems like the overhead of cross-thread communication would severely outweigh a simple synchronous lock call.
Then again, if you can do things like background non-essential operations, then the higher-level benefits can probably outweigh that.
Like I said, GCD has a heavy emphasis on performance. For example, a dispatch_sync will involve no cross-thread communication in the uncontended case, and the cost is comparable to taking a spinlock. A bit heavier, but not too much.
dispatch_async will necessarily be slower, but don't overestimate how much work is really going to happen in the cases where it matters.