October 24, 2014

Swift, Reflection, Downcasting, Protocols, and Structs: A solution

Posted in iPhone development, mac development tagged , , , at 10:01 pm by tetontech

Currently, Swift does not have the ability to do full reflection. It can, through the mirror struct, examine property names and values. Swift does not allow you to set the values directly. You can, however, instantiate a struct based on the structs type but the compiler will crash if you attempt to do this in any anonymous way. In addition to no real reflection abilities, Swift objects and structs do not support key-value coding. It turns out that a solution to some of the reflection issues in Swift is the addition of key-value coding to structs. An additional restriction to doing reflection in Swift is the inability to downcast an instance who’s type Any or AnyObject to a known protocol. It may be that Apple is resolving these issues but that doesn’t help those of us who need to do reflection now. Let’s look at what is available in Swift. This will show us its current limitations.

The reflect function and MirrorType

The reflect function and MirrorType structure were designed by Apple to display information for the Xcode IDE. Xcode uses these to show debugging information. This is why the results are read-only. The example below shows a struct being examined. The struct is not complex. It represents a person and has name, age, optional height, and calculated description properties.

struct Person:Printable{
    var name:String
    var age:Int
    var height:Double?
    var description:String {
        return "name:\(self.name) age:\(self.age) height:\(self.height)"
    }
}

Using the standard MirrorType and the reflection function we can get almost all of the properties and their values. Unfortunately there are problems accessing both the description property and working with the height. The description doesn’t show up in the children and the value returned for the height is an optional.

let aPerson = Person(name:"Sally", age:35, height:5.9)
let structMirror = reflect(aPerson)
let numChildren = structMirror.count
println("child count:\(numChildren)")
for index in 0..<numChildren{
   let (propertyName, propertyMirror) = structMirror[index]
   println("name: \(propertyName) value: \(propertyMirror.value)")
}

Run Results:

child count:3
name: name value: Sally
name: age value: 35
name: height value: Optional(5.9)

 

The height being an optional would not be a problem except there is no way to ask if something is an optional. If there was, then we could eventually get the underlying value. Since there isn’t a way to detect an optional I could not use this approach in my project. I needed the actual value. I needed it to corrctly building a piece of Sqlite insertion SQL. You will probably run into the same limitation in your projects.

Instantiation

In Swift an instance can be created from the struct or object’s description. Doing this to create a new Person is shown below.

let child = Person.self(name: "bob", age: 3, height: 2.5)

This works great as long as you know ahead of time what type of instance needs to be created. It is possible to store the Person.self value, it’s type is Metatype, in a constant, variable, or collection and create an instance using the variable.

let personType = Person.self
let personFromType = personType(name: "jessie", age: 14, height: 5.2)

Creating collections of different types of Metatypes, a feature needed in most reflection based coding, is also possible. It does require that each struct implement a common protocol. For this example I’ve created the Thing protocol. It declares that any Thing must have a name of type string.

protocol Thing{
    var name:String { get set }
}

I’ve modified the Person struct to implement Thing and created a Dog struct that is also a Thing.

struct Person:Printable,Thing{
    var name:String
    var age:Int
    var height:Double?
    var description:String {
        return "name:\(self.name) age:\(self.age) height:\(self.height)"
    }
}
struct Dog:Thing{
    var name:String
    var breed:String
}
protocol Thing{
    var name:String { get set }
}

Now an Array or a Dictionary can be created that holds both Person and Dog Metatypes.

let initializerList:[Thing.Type] = [Person.self, Dog.self]
let initializerDict:[String:Thing.Type] = ["Person":Person.self, "Dog":Dog.self]

It seems strait forward to initialize a Person. The following code seems like it should work but fails to compile.

let aMetaType = initializerDict["Person"]!
let anotherPerson = aMetatype()

The compiler failure says that a Thing doesn’t have an init function. That is true. The Thing protocol has no init function with no parameters defined. We can add one and modify the structs to also have init functions with no parameters. Now the compiler error goes away, but to do this well all the properties of the structs have to either be Optionals or have meaningful default values. Changing the properties to Optionals causes the problem mentioned earlier regarding getting actual values out of Optionals without know what type they are.

Adding an init to the Thing protocol causes another problem. Initializing a struct from the stored Metatype causes the compiler to segmentation fault (crash). The compiler should have either failed to compile the code since Thing doesn’t have an actual init function or a virtual lookup should have been done to call the Person’s init function.

So we have hit three dead ends using Swift’s standard behavior. We can’t get actual values from optional properties, we can’t create instances from commonly stored Metatypes, and the compiler crashes when we try to do anything complex.

Another reflection issue in Swift is the inability to downcast from Any to a custom protocol type, another common need when using reflection in code. You can downcast to a standard type like String (anAny as String) and even a custom type like Person (anAny as Person). But there is a compilation failure if you try to downcast to Thing (anAny as Thing). The error states that Any and Thing are unrelated. This indicates that custom protocols are not an Any.

What useful reflection can be done under these limitations?

A Solution

As mentioned in a previous post, I’m writing a library to work with Swift structs in the same way that CoreData works with objects. To create this library I  must do reflection. I need to convert structs into SQL statements and Sqlite result sets into Arrays of structs who’s types are unknown. To get around Swift’s current reflection limitations I applied key-value coding. The custom protocol KeyValueCodable makes this possible.

protocol KeyValueCodable{
    var KVTypeName:String {get}
    init()
    subscript(index:String)->Any? { get set }
    func instantiate()->KeyValueCodable
    func downCastFromAny(anAny:Any)->KeyValueCodable?
}

KeyValueCodable includes an initializer without any parameters, but it is only used by KeyValueCodable‘s instantiate function so we avoid the compiler crashing problem. It overcomes the issue of retrieving actual values from optionals by having a subscript that can return and set the values of any type of property. It overcomes the down casting problem by having each struct do its own downcast. It also overcomes a previously unmentioned reflection problem. You can’t get a string representation of a struct’s name from Swift’s standard reflection behavior.

Using KeyValueCodable puts more responsibility on the programmer than is usually required in other languages, but it makes both reflection and key-value coding possible. Here is the Person struct modified to implement the KeyValueCodable protocol. There is much more code in Person than there was before, but most of it is boiler-plate code. It can be copied, pasted, and modified.

struct Person:Printable,KeyValueCodable{
    let KVTypeName = "Person"
    var name:String?
    var age:Int?
    var height:Double?
    var description:String {
        return "name:\(self.name) age:\(self.age) height:\(self.height)"
    }
    init(){}
    func instantiate() -> KeyValueCodable {
        return Person()
    }
    func downCastFromAny(anAny: Any) -> KeyValueCodable? {
        if isKVCodable(anAny){
            let mirror = reflect(anAny)
            let numChildren = mirror.count
            var aPerson = Person()
            for index in 0..<numChildren{
                 let (propertyName, propertyMirror) = mirror[index]
                 switch propertyName{
                 case "id":
                     aPerson["id"] = propertyMirror.value as? String
                 case "name":
                     aPerson["name"] = propertyMirror.value as? String
                 case "height":
                     aPerson["height"] = propertyMirror.value as? Double
                 case "age":
                     aPerson["age"] = propertyMirror.value as? Int
                 default:
                     0//do nothing
                 }
             }
             return aPerson
         }
         return nil
     }
     subscript(index:String) -> Any?{
        get{
            switch index{
            case "name":
                return name
            case "age":
                return age
            case "height":
                return height
            case "KVTypeName":
                return KVTypeName
            default:
                return nil
            }
        }
        set(aValue){
            switch index{
            case "name":
                name = aValue as? String
            case "age":
                age = aValue as? Int
            case "height":
                height = aValue as? Double
            default:
                0//do nothing
            }
        }
    }
}

Person now has a subscript which can be used to get and set the value of any parameter. This is also true when the Person has been upcast to a KeyValueCodable. Person can now be ‘downcast’ from an Any. This is done by creating a copy of the data in the Any and applying it to a new Person. This isn’t such a big deal since Swift structs are copied every time they are passed to a function anyway.

There is a function, isKVCodable, called in the downCastFromAny function. It uses reflect and MirrorType to look for the KVTypeName property required to be part of each KeyValueCodable.

func isKVCodable(possibleCodable:Any?)->Bool{
    if let anActualAny = possibleCodable?{
        let mirror = reflect(anActualAny)
        let numChildren = mirror.count
        var aCodable:KeyValueCodable?
        //discover if is a codable
        for index in 0..<numChildren{
            let (fieldName, fieldMirror) = mirror[index]
            if fieldName == "KVTypeName"{
                return true
            }
        }
    }
    return false
}

Using KeyValueCodable I’ve been able to overcome Swift’s limitation for my reflection heavy project. I look forward to making it even more generic and useful for everyone. Suggestions are appreciated.

In another posting I will describe how to use reflection to find and access a struct’s methods.

 

October 22, 2014

Swift, C libraries, and Mapping Swift types to C pointer types

Posted in iPhone development, mac development tagged , , , , at 11:02 pm by tetontech

In my last posting I described a swift library, SwiftlyDB, I wrote to insert data into a SQLite database using the sqlite3 library. I promised some ‘lessons learned’ postings about what I picked up during the creation of that library. This is the first of those postings.

In Swift and Objective-C SQLite is accessed using a C library. This means a whole series of pointers are needed to move data back and forth between Swift and C. Thankfully Xcode will handle most, but not all, required Swift-to-C pointer conversions for you. When it doesn’t You will need to get used to COpaquePointer, UnsafePointer, and UnsafeMutablePointer. These Swift types represent C pointers, const pointers, and pointer pointers (**) respectively.

COpaquePointer

COpaquePointers represent pointers to any non-standard C data type. They can be a simple pointer or a pointer to an array, but they are not used to point to int, char, double, and other primitive types. When creating a Swift variable that refers to a sqlite instance, a COpaquePointer is used since a sqlite3 pointer is not a primitive type such as int or char. I assume the Swift pointer is called an opaque pointer since there is no way to tell what kind of C construct it represents just by looking at it. If you can’t see through glass to what is behind it,  the glass is opaque, right?

One result of calling sqlite3_open is the population of a COpaquePointer with the sqlite instance. This is done by passing the opaque pointer to the sqlite3_open function. The following two-line example shows a pointer being created and passed to the that function.

    var theDB:COpaquePointer = nil
    sqlite3_open((fileLocation as NSString).
                          cStringUsingEncoding(NSUTF8StringEncoding), &theDB)

Not so bad really. There is that & operator confusing the situation, and the conversion of an NSString to a C string to muddy the waters, but all in all, not too bad….until you look at the signature for sqlite3_open function.

UnsafeMutablePointer

Swift function signature – sqlite3_open(filename: UnsafePointer<Int8>,

ppDb: UnsafeMutablePointer<COpaquePointer>) -> Int

C function signature – int sqlite3_open(const char *filename, sqlite3 **ppDb )

Now things look much messier. The second parameter in the Swift signature seems to be a mess. The clue to figuring out what it means is the parameter name, ppDb. ppDb stands for Database pointer pointer. So the UnsafeMutablePointer<COpaquePointer> declaration must match up with the sqlite3** type. In the case of sqlite3_open, and in many other C functions, pointer pointers (**) are used to retrieve pointers to items generated during the function’s execution that are not returned. The UnsafeMutablePointer name was selected to reflect the in-and-out  nature of the C pointers represented by UnsafeMutablePointers. The pointer is unsafe since it might be null, and it is mutable since what the pointer is pointing at can change (it starts out pointing at null and can end up pointing at a swift3 instance). Swift provides the & operator to generate UnsafeMutablePointers from COpaquePointers. Don’t try to instantiate one yourself. Using the init method of UnsafeMutablePointer didn’t work at the time I wrote the library.

UnsafePointer

The sibling to the changeable UnsafeMutablePointer is the unchangeable UnsafePointer. In C terminology these are constant (const) pointers. The value of what they pointing at can be changed, but they can not be changed to point at a different memory location. The first parameter of sqlite3_open, seen in the Swift function signature above, is an UnsafePointer to an 8bit int. This must match the C parameter type of const char *. That is confusing until you understand a C character is the same thing as a C 8bit int.

When dealing with any C library, keeping these mappings between C and Swift pointer types in mind makes Swift-to-C library calls much easier.

October 14, 2014

Swift, Structs, and Data Storage/Retrieval with sqlite

Posted in Uncategorized at 8:52 pm by tetontech

A data storage and interaction problem exits in Swift. If someone uses structs instead of objects in an attempt to take a more functional approach in coding their application, CoreData won’t work. It only interacts with objects, not structs. While Apple may solve this in the future, that possibility doesn’t help us now. To resolve this issue I decided to create two libraries that would behave like an ORM (Object Relational Mapper) but work with structs (STRUM?).

The first library runs all interactions with sqlite3 on background threads so we don’t have to deal with threading. The sqlite3 interaction is SQL based and generates and uses prepared statements, transactions, and rollbacks. I call this first library SwiftlyDB and its source code is available on my github repository. It is MIT licensed and the main purpose of this posting is discussing its API. Later postings will cover Swift lessons learned during SwiftlyDB’s creation.

The second library, currently in the design stage, will work with structs much like any ORM works with objects. It will translate between the results returned by SwiftlyDB queries and create the appropriate structs. It will also generate the SQL required to store struct information using SwiftlyDB. This means that this second library, which I’m currently calling SwiftlyStore, will make it possible to safely store, update, retrieve, and delete from storage Swift structs in an intelligent and parallel way.

Now let’s get down to how to use SwiftlyDB. SwiftlyDB is functional, parallel, light weight (less than 350 lines of code), flexible, and easy to use. Its API consists of two structs, SwiftlyDb and DBAccessError, and four functions; setupSwiftly, discardSwiftly, swiftlyTransact, and swiftlyTransactAll. The first two functions setup and disconnect from a sqlite3 instance for you and the last two execute SQL.

To get started, use setupSwiftly to generate an instance of SwiftlyDB. You do this by passing it the name of an sqlite file. This example uses ‘test.sqlite.’

let (error,aSwiftlyDB) = setupSwiftly(“test.sqlite”)

If the sqlite file exists in your application’s bundle, setupSwiftly will copy it, only once, to your application’s documents directory so it can be used. If you choose not to ship a database with your app setupSwiftly will create the sqlite file in the documents directory for you. Either way, you end up with a usable sqlite database file for your app.

In the example line of code above, aSwiftlyDB is of type SwiftlyDB optional and error is of type DBAccessError optional. When your file is copied and the sqlite3 database is opened, error will be nil and aSwiftlyDB will not.

‘test.sqlite’ used in this example has only one table, the ‘dog’ table with id:text, age:integer, and height:double fields. To add a single dog use the swiftlyTransact function, swiftlyTransact(aSwiftlyDB:SwiftlyDB, sql:String, parameters:Array<Storable>?, resultHandler:(DBAccessError?, Any?) ->()).

The example below shows how to add a dog without using prepared statements which, by the way, is not usually a good idea. An SQL string is assembled to insert the dog and the string, aSwiftlyDB returned from the call to setupSwiftly, and a closure containing code to execute after the insertion is done are passed in as parameters. As is possible in Swift, the closure is placed after the closing parenthesis to aid in readability.

let insertString = "INSERT INTO dog VALUES (\"\(NSUUID.UUID().UUIDString)\", 5, 3.2)"
swiftlyTransact(aSwiftlyDB!, insertString, nil){(error:DBAccessError?, data:Any?) ->() in
    println("inserted")
    if let theErrorDescription = error?.description{
        println("oops. got an insertion error. \(theErrorDescription)")
        return
    }
    else{
        println("updated \(data!) records")
   }
}

After the insertion is done using the sqlite3 C library, the result of the insertion operation is passed to the resultHandler closure. In this case, the data parameter is a Int representing how many records were changed (1). I’ve shown some simple error handling made possible by having both an error and the transaction’s data as parameters to the resultHandler.

Now that data is in the database how do we get it out? A second call to swiftlyTransact will do that for us. In the code below, all of the dogs in the table are requested and printed to the console. You could present this to the user via your app’s user interface but I’m trying to keep the example simple.

let selectString = "SELECT * FROM dog"
swiftlyTransact(aSwiftlyDB!, selectString, nil){(error:DBAccessError?, data:Any?) ->() in
    println("selection")

    if let theErrorDescription = error?.errorDescription{
        println("oops. got a selection error. \(theErrorDescription)")
        return
    }
    else if let dogs = data as? Array<Dictionary<String,String>>{
        println("found \(dogs.count) in dogs")
        for dog in dogs{
            for fieldName in dog.keys{
                println("\t\(fieldName) : \(dog[fieldName]!)")
            }
       }
       println("done with dogs")
    }
}

swiftlyTransact also works well for single prepared statements. This example shows how to add a single dog like the previous insert example.

let preparedStatementString = "INSERT INTO dog VALUES (?,?,?)"
swiftlyTransact(aSwiftlyDB!, preparedStatementString, [NSUUID().UUIDString, 5, 3.2]){(error:DBAccessError?, data:Any?) ->() in
    println("inserted")
    if let theErrorDescription = error?.errorDescription{
        println("oops. got an insertion error. \(theErrorDescription)")
        return
    }
    else{
        println("updated \(data!) records")
    }
}

Any library that forced you to execute a function call for every SQL statement would be poorly designed. SwiftlyDB uses swiftlyTransactAll to fill the roll of executing ‘sql scripts’, swiftlyTransactAll(aSwiftlyDB:SwiftlyDB, tasks:Array<Dictionary<String,Array<Storable>?>>,resultHandler:((DBAccessError?, [Any]?) ->())?) -> ().

swiftlyTransactAll has three parameters, an instance of SwiftlyDB to work with, a list of tasks to do, and a closure to execute once the list of tasks is complete.

This example adds 10 random dogs to the dog table using swiftlyTransactAll.

var taskList = Array<Dictionary<String,Array<Storable>?>>()
for index in 0..<10{
    let parameters:Array<Storable>? = [NSUUID().UUIDString,Int(arc4random()%12),Double(arc4random()%100)/10]
    let aTask = [preparedStatementString:parameters]
    taskList.append(aTask)
}

swiftlyTransactAll(aSwiftlyDB!,taskList){(error:DBAccessError?, data:[AnyO]?) -> () in
    println("inserted")
    if let theErrorDescription = error?.description{
        println("oops. got an insertion error. \(theErrorDescription)")
        return
    }
    else{
        println("updated \(data!.count) records")
    }
}

The last function in the API closes the sqlite database for you and cleans up.

Example:

discardSwiftly(aSwiftlyDB)

So that’s it. The entire API. Most of the code exists in the SwiftlyDB.swift file, but one item of interest exists in the Storable.swift file. It would be good to limit the parameters to types of things that make database sense; Strings, Ints, and Doubles. To this end String, Int, and Double have been extended to implement a custom Storable protocol found in the Storable.swift file. This way types of things that don’t make database sense such as user interface elements, controllers, delegates, etc. can not be accidentally sent to one of the swiftlyTransact functions.

SwiftlyDB is available for your use. It is functional in nature. It handles sqlite database transactions and rollbacks for you. It executes all database interactions in parallel to your code and does this safely. Take it and use it in any way you want.

My next postings will look at Swift lessons learned as I implemented this library.

%d bloggers like this: