Tag Archives: swift

Now we’re all Forked!

Replicating Types, , , ,

TLDR; I’m launching a new Swift framework called Forked for working with shared data, both on a single device, and across many.

A few years ago, I was knee-deep developing the collaboration feature of our app Agenda. Agenda is mostly local-first, so it was a challenge. Effectively, Agenda is a decentralized system, and the collaboration feature would allow anyone in a group to edit a shared note at any time — even when they were offline for days. When each copy of a shared note was transferred over to the devices of other members of the group, the result had to be consistent. It would be unacceptable for two people to end up with different versions.

I mentioned that Agenda is a local-first app. That means there is no central server with any understanding of the data model, taking care of conflicts — there is no central truth. Each Agenda client app has to take the data it gets from the cloud, make sense of it, and merge it in such a way that the result is the same as what other devices end up with, even if the data in question is days old.

What I realized back then is that this problem has already been solved very elegantly by a product that is extremely well-known and popular, and right under our noses. It’s called Git.

If you treat each copy of the Agenda data as something akin to the latest commit in the branch of a Git repository, you can use the same approach as Git to merging data. And Git works: developers can go hiking in Alaska, develop code completely offline, come back and merge their changes, and all is good with the world.

Back to Agenda: I decided the solution was a class called BranchedFile. My goals at the time were to create a simplified, embedded version of Git, that would operate on a single file. It would support branching, with main and auxiliary branches that could be used to handle concurrent changes to the file, and merging to reach eventual consistency.

The system should not require a complete history of changes, but keep enough versions of the data to facilitate the 3-way merging used in Git. With 3-way merging, you use the two recent conflicting versions, and compare to a common ancestor. The common ancestor is a copy of the file at the point the two branches diverged.

This approach worked well. I was able to come up with some fairly straightforward rules for which versions of the file I needed to keep around in order to fulfill a merge. All of this is implemented in BranchedFile. Agenda has been using this now for several years whenever two or more people want to collaboratively edit a note.

I hadn’t looked much at that code for several years, but that changed early in 2024. I attended the inaugural Local-First Conf in Berlin. I gave a short talk about Ensembles, which is the Core Data sync framework I have developed for more than 10 years ago, and then I watched the other talks. And I got inspired, and started to wonder: what if I could make my BranchedFile type more generic, and perhaps even turn it into a genuine modeling framework, like a mini version of SwiftData.

I started to dream:

  • It should use structs instead of classes
  • It should track changes in branches, and have 3-way merging
  • It should be possible just to store data with Codable
  • Where merging is an afterthought in many data modeling frameworks, this framework should support advanced merging, employing the latest Conflict-free Replicated Data Types (CRDTs)
  • It should be possible to sync via iCloud and other cloud services with no change to the model
  • It should be useful not only for sync, but even for subsystems within an app on a single device

Today the dream has been fulfilled, at least up to the point of an MVP.

Today, I’m launching Forked, a new approach to working with shared data in Swift. And it has actually worked out better than I expected. I wasn’t even sure it would be possible to build, but with the new Swift macros, I was able to come up with a minimal API that seems to work great. I’m really looking forward to dog fooding it.

Let’s just finish up with a little code, so you can see how simple it turned out to be. Here’s a model from the Forkers sample app, which is basically a basic contacts app:

@ForkedModel
struct Forkers: Codable {
    @Merged(using: .arrayOfIdentifiableMerge) 
    var forkers: [Forker] = []
}

@ForkedModel
struct Forker: Identifiable, Codable, Hashable {
    var id: UUID = .init()
    var firstName: String = ""
    var lastName: String = ""
    var company: String = ""
    var birthday: Date?
    var email: String = ""
    var category: ForkerCategory?
    var color: ForkerColor?
    @Merged var balance: Balance = .init()
    @Merged var notes: String = ""
    @Merged var tags: Set<String> = []
}

What I love the most about Forked models is that they are just simple value types. The @ForkedModel macro doesn’t change the properties at all, it just adds some code in an extension to support 3-way merging. So you can use this on any struct, and the result can do everything your original struct could do, from encoding to JSON, to jumping seamlessly between isolation domains in Swift 6.

The merging that @ForkedModel provides is pretty powerful. It does property-wise merging of structs, and if you attach the @Merged attribute, you can add your own custom merging logic, or use the advanced algorithms built in (like CRDTs).

To give an example, the notes property above is a String. With @Merged applied, it gets a hidden power — it can resolve conflicts in a more natural way. Rather than discarding one set of changes, or merging to give somewhat arbitrary results, it produces a result a person would likely expect. For example, if we begin with the text “pretty cool”, and change the text to “Pretty Cool” on one device, and to “pretty cool!!!” on another, the merged result result will be “Pretty Cool!!!”. Nuff said.

And this works within your app’s process, between processes (eg with sharing extensions), and even between devices via iCloud.

Also worth noting: Forked models work great with Swift 6 structured concurrency, helping to avoid race conditions. When there is a chance you might get a race condition (eg due to interleaving in an actor), you can setup a QuickFork — equivalent to an in-memory Git repo — and use branches (known as forks in Forked) to isolate each set of changes, merging later to get a valid result.

To finish off, consider this: With your model supporting 3-way merging, it knows how to merge itself. All it needs is a conflicting version, and a common ancestor, and Boom! So adding support for CloudKit to your app is next to trivial, and your model can remain completely unchanged. Here is the code that Forkers uses to setup CloudKit sync:

let forkedModel = try ForkedResource(repository: repo)
let cloudKitExchange = try .init(id: "Forkers", 
    forkedResource: forkedModel)

// Listen for incoming changes from CloudKit
Task {
    for await change in forkedModel.changeStream 
        where change.fork == .main &&
              change.mergingFork == .cloudKit {
        // Update UI...
    }
}

That’s all of it! We just added sync to our app in less than 10 lines of code. Decentralized systems can sometimes be astounding, and they also work great even when your use case is not technically decentralized!

The Danger of Playing it Safe

SwiftUI

It’s an old chestnut that Swift developers love to sink their teeth into: Should you force unwrap optionals? Everyone has an opinion about it, and I’m going to state mine as clearly as I can — forcefully unwrap early and often.

Of course, this is just one opinion. We even disagree about it within our two man Agenda team, so there is probably not much hope for the world. But I want to explain when and why I think it is good to force unwrap, and give a recent real world example that demonstrates the cost of not doing it.

First, my reasoning. I think you should force unwrap in any situation where logic tells you that an optional simply cannot be nil. A standard example might look like this.

var count: Int?
count = 0
print("\(count!)")

Could count end up being nil in this case? Perhaps, through some strange memory corruption or solar flare. Do I want my app to continue to run if count is nil? No. To put it bluntly, that is cancer. The app would be in an undefined state, which may well be dangerous. I don’t want it deleting user data or causing any other havoc — it should just crash. I will be informed of the problem, and it can’t do any more damage.

But what is wrong with the softly, softly approach? Shouldn’t we be defensive here? Here is how you could do it.

var count: Int?
count = 0
if let count = count {
    print("\(count)")
} else {
    print("error!")
}

There are several problems with this. Firstly, it is more verbose, making it more difficult to read. Secondly, and even more importantly, it disguises developer intentions. It says to the next developer “count could conceivably be nil, so we will check for it”, even though it was clear to the original developer that it should never be nil in any well-defined circumstance. Lastly, where is the app at after this code runs? Who knows? The state is completely unclear.

So be assertive with forced unwrapping. If there is no case where the optional should ever be nil, force unwrap. But don’t force unwrap in cases where there is a valid situation in which nil could arise. If a foreseen error could lead to nil, that requires explicit handling.

I mentioned that I wanted to finish off with a recent real-world example. When Apple announced iOS 14 would ship, they only gave about a day’s notice, leaving most of us scrambling to get ready. For Agenda, we had done a lot of work on widgets, but we had never had them in a beta, or extensively tested on the device. That was planned for the week or so that we usually have following release of the Xcode GM, but before the new iOS is available. Alas, in 2020, that was a luxury we didn’t have.

So I merged our branches in Git, and uploaded the binary. Downloaded from TestFlight, and — you guessed it — the widgets were broken. It was just showing a placeholder, which I later learned is what you get when a widget crashes.

I dug into the code, and I found that the problem was an incorrect file path. I had been testing with a special development target, and hadn’t properly tested the production target. The faulty code was as follows:

let identifierSuffix = (Bundle.main.object(forInfoDictionaryKey: "AgendaIdentifierSuffix") as? String) ?? ".development"

It is attempting to get a string from the Info.plist, which it will then use in the aforementioned path, but I had forgotten to put the string in the Info.plist. That wasn’t the real crime though: As you can see, in the case of nil being returned, it tries to play it safe by falling back on the development target’s string.

This was bad. It meant the production app had the wrong path, so it crashed, but I didn’t see the crash until very late in the development cycle, even though it was there the whole time. And the cost? I probably wasted an hour or two uploading the binary, then sluthing the problem. “What’s an hour or two?” I hear you ask. Well, Apple were pushing us to submit our new widget feature so that Agenda could be considered for featuring. I don’t think two hours was enough to jeopardize that, but you never know. I’m sure the App Store is a madhouse on release day.

Here is how I think the code should have been written.

let identifierSuffix = (Bundle.main.object(forInfoDictionaryKey: "AgendaIdentifierSuffix") as? String)!

It is simpler, clearly stating that I am assuming the entry is in the Info.plist, and the widget only works if that assumption is fulfilled. If there is no entry in there, the widget is entirely useless, and should crash before causing more damage. And, with this code, it will crash early in development. I would have discovered the issue as soon as I tried to run a debug version, and fixed it long before it hit the App Store.

(Update: It has been pointed out that the last line of code is a bit convoluted. It would read better as

let identifierSuffix = Bundle.main.object(forInfoDictionaryKey: "AgendaIdentifierSuffix") as! String

That’s very true. I wanted to change the original code as little as possible, to stay on message. The point is the same: defensive coding did more damage than good, and an assertive force-unwrap would have been better.)