February 1, 2016

Swiftly Idle

Posted in Uncategorized at 9:00 pm by tetontech

With the release of the open source Swift compiler, it becomes feasible to write Swift server-side apps for Linux. Server-side apps require two basic components, parallel processing (threading), and the ability to never exit.

While these two requirements are not scheduled to be fully ready for the Linux version of the compiler until Swift 3.0, I wanted to get ahead of the curve and start experimenting now. To do this I decided to design and create, using the Swift 2.2 open source compiler on OS X, an app I’m calling Courier.

Since Courier is intended to run constantly without consuming CPU resources I thought I’d share a code example that keeps the main thread in an idle state but allows the main thread to respond to dispatches from GCD. This fundamental ability is the basis for all servers and other types of monitoring apps…even GUI apps and OS’s.

Below is the code for a function I’m calling waitAndListen. It will setup listening for changes to one of the server’s directories. When a change is detected, it will report what happened.

To make this work, all the listening has to be set up early in the source code. To make the waitAndListen example easier to understand, I’ve moved the code for setting up listening into a function called watchDirectory. We’ll look at that function a little later.

1 func waitAndListen(){
2    //call a custom function to watch for changes in the "stuff" directory.
3    let source = watchDirectory("stuff")
4    //setup the current thread's runloop in an infinite loop 
5    //to keep the app going
6    while(true){
7        CFRunLoopRunInMode(kCFRunLoopDefaultMode, 3600.0, true)
8        print("looping")
9   }
10 }

Line 6 is where the good stuff starts. It is the beginning of an infinite loop. Don’t worry. It won’t soak up your CPU cycles. Line 7 makes sure of that. Line 7 is a special function call. What it does is make a OS blocking call that returns every hour (the second parameter) or when the ‘stuff’ directory is changed, which ever comes first. Until then it sits and waits. The first argument isn’t particularly important to this discussion, but the third parameter, a boolean, is used to indicate if CFRunLoopRunInMode should exit after it is interrupted. I’ve set it to true for this example to keep things simpler.

There are more complicated examples of using CFRunLoopRunInMode on the web. Those usually check the return value of CFRunLoopRunInMode to determine if the application should exit. Since I’m writing something that should never exit of its own choice, that kind of check is unneeded.

In short, lines 6 – 9 are all that is needed to create an application designed to never exit normally.

One nice thing about using CFRunLoopRunIn Mode to put the main thread into a waiting state is the main thread can still be used for computation. The watchDirectory function, called in line 3, makes the application to wake up the main thread and use it when changes are made to the ‘stuff’ directory. watchDirectory is longer and more complicated than the code in waitAndListen. We’ll take it a step at a time.

In line 2 below, a file descriptor is created. This descriptor refers to the ‘stuff’ directory and is used later as part of the code required to listen for changes to the ‘stuff’ directory.

Line 4 is where we get a reference to the main thread. If you don’t want to use the main thread for handling changes you can use GCD’s dispatch_get_global_queue instead. If you choose to use the global queue rather than the main thread, the main thread will never be wakened by events. Instead, the background thread will become active each time a change is made.

Grand Central Dispatch (GCD) is a dispatching engine. Line 5 adds a new, non-preexisting event source to GCD. The _VNODE constant tells GCD the type of events it should dispatch are file and directory events. The descriptor for the directory we want to watch is passed as the second parameter so GCD knows which directory to monitor.

The third parameter consists of a series of constants that are assembled using swift’s bitwise OR operator. The options used in this example sets up the dispatch source to include all types of file and directory events. You can restrict these to a smaller set if you choose. Lastly, the queue to dispatch to is passed in as the last parameter.

With a dispatch source added to GCD, GCD needs to be told what to do when a change is made to the directory being watching. That is done by adding the closure beginning on line 7 as a dispatch handler.

1 func watchDirectory(directoryPath: String) -> OS_dispatch_source{
2    let directoryDescriptor = UInt(open(directoryPath, O_RDONLY))
3    //attach the listener to the main thread.
4    let queue = dispatch_get_main_queue()
5    let dispatchSource = dispatch_source_create(DISPATCH_SOURCE_TYPE_VNODE, 
                 directoryDescriptor, DISPATCH_VNODE_DELETE 
 6   //assign a closure to be the listener for directory change events
 7   dispatch_source_set_event_handler(dispatchSource){
 8       let eventType = dispatch_source_get_data(dispatchSource)
 9       //detect the different types of changes and handle them
 13          )
 14           {
 16               //handle removing and deleting
 17            }
 18      //when something is added or something is removed from this 
 19      //directory it is a WRITE
 21      {
 22           //handle a write
 23      }
 24  }
 25  dispatch_source_set_cancel_handler(dispatchSource){
 26       close(Int32(directoryDescriptor))
 27  }
 28  dispatch_resume(dispatchSource)
 29   return dispatchSource
 30 }

Lines 7 through 24 consists of the code GCD executes each time a change is made to the ‘stuff’ directory. The example above varies its behavior depending on the type of event. It handles deletions and renamings in one way and adding files and directories in another. Before it can handle the events differently, the code must determine the type of event that happened. To do this,  the dispatch_get_source_data function (line 8) is called each time a change is made.

Once eventType is set to the type of event that happened, the example uses if conditional statements to handle the various types of events. The event could be directory or file creation, deletions, renaming, etc. You can see the if statements for this example on lines 10 and 20.

There are a bunch of events associated with each if statement. Since I’m preparing to run this code on both OS X and Linux, I’ll need to experiment with the event types on Linux before I can decide exactly how to break up the types across my applications’s statements. I’m assuming there will be differences in how Linux handles these events compared to OS X.

If you want to cancel watching the directory, the dispatch_source_set_cancel_handler must be called. The example shows that function cleaning up the file descriptor that refers to the directory.

Line 28 is very important. If you don’t resume/restart the dispatching engine, no events will ever be captured and dispatched to the listener you’ve areated. In other words, your code for handling the changes to the directory won’t be called when the directory is changed.

Overall the API for watching a directory or file isn’t too bad.

Now….back to creating Courier. :)


November 12, 2015

Adding Custom Class Properties with Swift Extensions

Posted in Uncategorized at 10:59 pm by tetontech

The previous posting, Functionally Dreaming with Swift, was an exploration of what is possible when coding for iOS and OS X. As part of the code example I ran across the need to uniquely identify any given user interface element in Interface Builder (IB). One commonly explored way to do this is to use the restoration ID of a UIView as a unique identifier. But this has nothing to do with restoration id’s regular use. Using it for uniqueness but not restoration behavior doesn’t pass ‘the smell test.’ It abuses an existing property and can cause restoration behavior when it is not wanted.

What, then, is a poor Swift programmer to do? Swift extensions can’t add stored properties.

There is a solution. IB has the ability to apply Key-Value coding to any User Interface (UI) element but only for existing properties. Somehow stored properties have to be mocked-up so we can take advantage of IB’s Key-Value coding application. This is actually easier than one would think.

Let’s start by dropping IB from the picture so we can keep things simple. Imagine for some reason you wanted to extend all UIViews (this would include UIButtons, UILabels, UIImageViews, etc.) to have a stored property called sillyName. If all went well then you would be able to set the sillyName attribute in the ViewController’s viewDidLoad method.

Image 1: Setting a property added by an extension.

Image 1: Setting a property added by an extension.

Image 1 shows how this would look. Since sillyName would be like any other property it would be set in Swift’s normal way. You would also get the value by putting

let aSillyName = self.view.sillyName

in your code. The way to make this possible it to mock up a custom property.

In Swift, private declarations of structs is allowed in many places. Extensions is one of those places. Let’s draw on that and Swift’s ability to create calculated extension properties. Image 2 shows how to mock up a sillyName stored property as part of an extension.

Image 2: The source code for mocking up a stored property.

Image 2: The source code for mocking up a stored property.

Lines 27 – 29 embed a private struct into an extension. It is important to understand that this is different than attempting to add a named value (let) or a variable (var) to UIView as a part of the extension. In this case there is no instance of SillyCustomProperties created.

Instead of using an instance, we will use it ‘statically.’ This allows us to write code like line 32. In that line of code, the static struct property is accessed and returned from the get method of the extension’s calculated sillyName property. The sillyName calculated property is defined on line 30 as a String Optional and the setter is defined on lines 34 – 36.

This is a lot of fun, but there is a significant problem with the code in Image 2. The struct and its static values are shared among all UIView instances. That means that there could only be one sillyName and it would be applied to ALL UIViews. Each time it was set it would be updated for all UIViews. This means that we don’t yet have the ability to use this approach to apply unique identifiers to UIViews. If we tried they would all share the same ID. That is NOT good. To solve this problem we need to apply a little Objective-C ‘magic’.

Since UIView’s are instances of Classes and custom key-value pairs CAN be added to any class instance, we can ‘calculate’ a Swift property by storing and retrieving a value using Objective-C Key-Value coding. This can look a little nasty since we will need to call C functions and pass pointers.

Image 3: Mocking stored properties with unique values.

Image 3: Mocking stored properties with unique values.

Image 3 contains the ‘magic.’ Let’s replace the bodies of the get and set methods. In the new get method body, the objc_getAssociatedObject function retrieves the value associated with a custom key using Objective-C Key-Value coding. The function has two parameters. The first is the class instance that has the custom key and value. In this example it will be a UIView since we passed self and UIView is what we have extended.

The second parameter is the key for which we want the value. Line 40 shows the NSString pointer to the memory location of sillyName being passed. The & is what forces the parameter to be a pointer. NSString is inferred by the compiler since objc_getAssociatedObject requires an void*, C’s version of Swift’s AnyObject type, as its second parameter. If you would like more information on how to interact with C functions there is an earlier post about that.

The objc_setAssociatedObject is similar to objc_getAssociedObject but has one additional parameter, an indicator stating  the memory for the NSString created as a result of calling the function should be retained rather than released. This reserves the memory used for the NSString until we decide to get rid of it  (I.E. replace the value associated with sillyName with some other string). Now we can assign unique values to any type of UIView in interface builder.

Image 4: Setting Key-Value pairs using Interface Builder.

Image 4: Setting Key-Value pairs using Interface Builder.

Image 3 shows a UIImageView and the Identity Inspector’s (the blue icon in the Utilities list) description of the view. The section titled, “User Defined Runtime Attributes” is IB’s location where you can assign values to the properties of the UIView. In this case the value ‘squiggles’ is assigned to the sillyName property. Since sillyName was mocked up as a stored property of all UIViews by our extension, this definition does not cause a compile-time failure like it would if we hadn’t mocked up sillyName.

With all these pieces in place, we can modify this approach so we can meet the need for unique identifiers for any view. The code for this is almost exactly the same as what you have already seen. As seen in Image 5, the only differences are the name of the Struct and the name of the struct’s String property. Ignore the Bool properties. They are for something else entirely.

Image 5: Mocking up SwiftlyFunctional's uniqueID

Image 5: Mocking up SwiftlyFunctional’s uniqueID

So there you are. This pattern can be used to add ‘stored’ properties to classes. I you want to see how the Unique ID was set for this code look in the Functionally Dreaming with Swift posting.

November 10, 2015

Functionally Dreaming with Swift

Posted in Uncategorized at 5:16 pm by tetontech

Making iOS or OS X apps using a functional programming approach means dealing with a lot of application pieces designed using an Object Oriented (OO) approach if you use the default templates and libraries. The user interface (UI) and the data transmission and storage behaviors for both OS’s are heavily object oriented. This dramatically restricts the space where functional programming designs and techniques can be applied.

Having experienced this restriction I decided to explore what iOS development would be like if the UI behaviors were functional rather than OO in their design. Swift, much more than Objective-C, enables this kind of exploration so I created a Swift example of what could be done. I call it SwiftlyFunctional and an example Xcode project is in my gitHub repository.

Swift uses pass-by-reference for instances of classes and pass by value for structs. To have a ‘truly’ functional approach the UI Elements of iOS and OS X would have to be changed to be structs. That isn’t going to happen and I have no desire to write struct wrappers for the many different types of UI elements in iOS and OS X. It wouldn’t be impossible, but is outside of the scope of what I wanted to explore. I’m chose to ignore this problem as part of my dreaming.

When laying out what I hoped to accomplish, I decided I still wanted to be able to use Interface Builder (IB) to design and layout the UI, but not be restricted to using the Object Oriented (OO) design enforced by the current Xcode templates. Because of this, I decided to dump working with the ApplicationDelegate and ViewController classes generated in a new iOS and OS X projects.  Instead of using the ViewController class to add IBAction Selectors to UI elements and gesture recognizers as is traditionally done, I  wanted to use Swift functions and closures to handle UI events.

With these goals in mind, I began my dreaming. What I show here is what I came up with but it is only one of many ways this could be done. I’ll show you how to use the SwiftlyFunctional code in this posting. In subsequent postings I’ll describe how some of the more interesting portions of the code work.

Image 1: The structure of a SwiftlyFunctional app

Image 1: The structure of a SwiftlyFunctional app

Let’s start with the structure of a functional iOS app. Image 1 shows that the ApplicationDelegate.swift and ViewController.swift files are gone. Instead you can see the SwiftlyFunc.swift and Main.swift files. You can call Main.swift anything you would like except for main.swift. The capitalization is important. If you try to use main.swift you will get a compilation error. This is likely due to the hidden files in all Swift apps that allow for Objective-C interactions. The file main.h is part of the default Objective-C projects.

Main.swift is where you will begin writing the code for your application. It contains the main(application:userView:launchReasons) function. This function acts somewhat similar to the C main function and is the where the app will start running your code.

the main function, as you can see in the example code, is where you can attach Swift functions or closures to application events such as ‘did become active’ and UI elements like buttons, images and labels.

In the example source code, and in Image 2, you can see an example of adding a closure for the ‘did become active’ application event.

Image 2: Attaching a closure to an Application level event.

The addApplicationEventHandler(application:theEventType) function is part of the SwiftlyFunctional API. It can be used to add a closure to any app event except application(didFinishLaunchingWithOptions:launchOptions). That one happens much earlier than will be captured and made available for closures. The addApplicationEventHandler function has three parameters, the application running (this is the first parameter of the main function), an enum for the type of application event to map to, and a closure with no parameters and a void return. In image two the closure only prints a string to the console, but you would be able to perform any computation normally associated with a ‘did become active’ event in iOS.

Image 3: The types of events that can be linked to closures.

Image 3: The types of events that can be linked to closures.

image 3 contains the complete list of Application events to which SwifltyFunctional can add closures or function. If other events are added by Apple they can be easily added to the enum.

Assigning closures to events for UIControl elements such as Buttons is nearly the same. One major difference is the closure must be associated with an event triggered by a specific UI element. In the case of the example, this is a UIButton. To make this possible, a reference to the button is needed. The SwiftlyFunctional API has a method, getViewByUniqueID(containingView:anID).

The first parameter passed to getViewByUniqueID in Image 4 is ‘userView.’ This UIView is the same as the second parameter of the main function and represents the topmost view of the view controller in the view hierarchy created in IB.

Image 4: Attaching closures to UIButton events.

The second parameter is the type of event, TouchDown and TouchUpInside for this code snippet. These event types are part of of the standard iOS library UIControlEvents struct. Like the addApplicationEventHandler function, the last parameter is the closure to activate when the event of touchEventType occurs.

Closures can also be attached to gestureRecognizers. In the SwiftlyFunctional example project users Interface Builder to add a TapGestureRecognizer to chalk.png’s UIImageView. The TapGestureRecognizer and the UIImageView both have a uinqueID added to them using KeyPaths. You can see how this is done by examining the right-hand side of Image 5.

Image 5: UITapGestureRecognizer for a UIImageView.

Set up this way, the SwiftlyFunctional’s getViewByUniqueID, getGestureRecognizerByUniqueID, and addTouchEventHandler functions can be used to find the recognizer and attach a closure to it (See Image 6).

Image 6: Attaching a closure to a UIGestureRecognizer event.

Unlike closures for UIControls such as UIButtons, no UIEvent object is passed to the closure by SwiftlyFunctional. UIGestureRecognizers contain the information obtainable from UIEvents. Notice that the location of the tap in the UIImageView is directly available without getting or using a Set of UITouches like you saw in the UIButton code snippet.

So there it is. Now closures can be attached to Application, UIControl, and UIGestureRecognizer events using a functional rather than an OO approach. Take a look at the example project, specifically the SwiftlyFunc.swift file, to see how this was done. I’ll follow up this posting with a couple of explanations of the SwiftlyFunctional code.

November 3, 2015

SwiftlyHybrid and AndyHybrid

Posted in Uncategorized at 8:28 pm by tetontech

In several of my previous posts over the last seven years I’ve described how to use web views with HTML5, CSS3, and JavaScript to develop installable apps for both iOS and Android. The earliest examples lead to the creation of QuickConnect (QC) and influenced the development of its first competitor, PhoneGap/Cordova. Those tools and others like them included extensive ‘bridging code’ to enable calls to Objective-C or Java and then back to JavaScript so the weaknesses of the older versions of the web views could be overcome.

As the iOS and Android web views’ capabilities have grown, the need to augment them with additional native iOS and Android code has decreased. Now audio and video can be recorded and played within the web views. Pictures can be taken and voices can be used to read text. Data can be easily stored and retrieved from local and remote stores. Nearly all of the functionality developers now need is encapsulated in the web views. Because of the continual growth of HTML5, CSS3, JavaScript, and web view support for them, heavy and complicated tools like QC and PhoneGap/Cordova are no longer needed. Instead much lighter and more standardized tools can be used. I’ve created SwiftlyHybrid and AndyHybrid to show what can be done. You’ll find each of them in their gitHub repositories. They are MIT licensed so feel free to do anything you like with them. SwiftlyHybrid has iOS and OS X example projects and AndyHybrid includes an Android Studio example project.

SwiftlyHybrid uses Apple’s WKWebView and the Swift programming language. AndyHybrid uses Google’s WebView that ships with Lollipop and Marshmallow. (Google claims this version of WebView will degrade gracefully when running on older Android OS versions.) Over time, the API’s for these two web views and their HTML5, CSS3, and JavaScript support have been converging. This means it is easier to write hybrid apps than ever before.

For both SwiftlyHybrid and AndyHybrid nearly all applications will only need to replace the HTML, CSS, and JavaScript files with custom ones and be done. For those few apps that still need to do something not possible in the web views’ sandbox I’ve created a standardized way to call from JavaScript to Swift or Android and get results back. HTML5 support and this bridge help SwiftlyHybrid and AndyHybrid use less RAM and CPU than the older QC and PhoneGap/Cordova tools, yet they are less complicated, and just as capable.

If you want to make calls down to Swift for iOS or Java for Android using one or both of these tools, the JavaScript code is the same. Here is a silly example that works for Both SwiftlyHybrid and AndyHybrid. In the example a message is put together and then past to the native code using the postMessage method.

var clicks = 0
function sendCount(){
    var message = {"cmd":"increment","count":clicks,
        var response = JSON.parse(responseAsJSON)
        clicks = response['count']
        document.querySelector("#messages_from_java").innerText = 
                                                  "Count is "+clicks

The message is a JavaScript associative array. it contains the command “increment” and some data, the current value of the “clicks” variable, and a callback function. Calling the postMessage function provided by both SwiftlyHybrid and AndyHybrid will convert the message associative array into JSON and send it to the native code portion of your app. In order for the callback function to be passed, it must be converted to a string otherwise the JSON library will strip it out. Since this message isn’t going across the internet or an internal network we don’t have to worry about XSS attacks so executing the string in the native code as JavaScript isn’t a security issue.

The Swift code for handling the message is in the SwiftlyMessageHandler.swift file in the SwiftlyHybrid example. It looks like this:

let command = sentData["cmd"] as! String
        var response = Dictionary<String,AnyObject>()
        if command == "increment"{
            guard var count = sentData["count"] as? Int else{
            response["count"] = count
        let callbackString = sentData["callbackFunc"] as? String
        sendResponse(response, callback: callbackString)

The current count is first pulled from the data sent from JavaScript. It is incremented and then SwiftlyHybrid’s sendResponse method is used to execute the callback JavaScript function using the response dictionary as a parameter to the JavaScript callback function.

The Java code for AndyHybrid is very similar. It is in the JavaScriptCommunication.java file.

HashMap<String, Object> message = (HashMap) JSONUtilities.parse(aMessageAsJSON);
String command = (String)message.get("cmd");
Serializable response = null;
       long count = (long)message.get("count");
       HashMap<String,Object> dataMap = new HashMap<>();
       response = dataMap;
String asyncCallback = (String) message.get("callbackFunc");
sendResponse(response, asyncCallback);

In both languages the command is retrieved from the associative array generated from the JSON message string, the count is also retrieved and incremented, a response is created containing the updated count, and the response is passed to the callback JavaScript function. The callback, in both cases, is executed within the web view and in the web view’s thread.

Having shown you the JavaScript, Swift, and Android Java to do communication between the two ‘layers’ of hybrid apps, I must say again, seldom is this type of inter-thread communication needed. Most of what you will want to do can be done directly in JavaScript, HTML, and CSS.

July 31, 2015

A Little Bit of Fun

Posted in Uncategorized at 9:24 pm by tetontech

I’ve recently put a free book up on Apple’s book store. It’s a little bit of fun I did in my ‘spare’ time. I call it “Doing Stuff With C.” Here is the URL. https://itunes.apple.com/us/book/doing-stuff-with-c/id1023155821?mt=11

I’ve tried to take a simplified, light-hearted approach to introducing C to someone who doesn’t know anything about it. I envision this as the first of a free book series, Doing Stuff With…., that covers the same material in each book but for different languages/platforms. I’m hoping to have the next book, Doing Stuff With Java, done next week.

I’d love to hear feedback on this little bit of fun and where to take the series next (I’m already planning Doing Stuff With Swift).

FYI, you will find other for-money books by me on Amazon and the other online stores. Don’t buy them. They are out of date. I did them through traditional publishers. That causes problems with keeping the information from going stale.

All of this writing AND working on Swiftly Secure??? Good thing I’m taking the next 5 weeks off from work!


Encryption development update

Posted in Uncategorized at 5:17 pm by tetontech

In order to continue with the development of my socialist millionaire protocol implementation that uses a 4096 bit key I need to find a generator for a Galois field of that size. This is no short term undertaking. Even using the access I have to a supercomputer it is going to take a while to find a generator.

The current level of mathematical understanding is that these generators must be brute-forced since there is no known pattern for what is and isn’t a Galois generator. So while my code is using the supercomputer to do a brute force search, I’m attacking the problem of finding a pattern. I’ve attempted to take a non-obvious approach and have already seen some interesting patterns. If what I’m seeing as patterns hold, I should be able to find a generator long before the brute force approach does. That would be nice since checking even one generator for 2^4096 values to see if the generator actually is a generator takes a long time, let alone multiple possible generators.

I’m eager to get the pattern found so I can share it with you and complete the SMP implementation.

January 20, 2015

Swiftly Secure, Part 1 – Large Number Multiplication

Posted in iPhone development, mac development, Uncategorized tagged , , , at 8:52 pm by tetontech

I have a swift iOS project where I want users to be able to communicate between their devices by sharing data via a server. I want their communications to be encrypted in such a way that no one can decrypt them except the two parties communicating. In fact, I want their communications to be encrypted end-to-end without them being de-cryptable when they are on the server. To accomplish this, I’ve decided on using the Socialist Millionaire protocol since it handles both encryption and authentication.

Doing encryption of this type means I’m going to need to multiply integers that are too large to fit in any of the standard Swift or Objective-C integers. I explored some existing large number libraries in C, but the most commonly used ones appear to be GPL or LGPL licensed. I won’t use libraries under those licenses in a closed source app so I decided to create something of my own.

In this posting I’m going to show you a part of what I’m creating — how to multiply numbers in swift without encountering overflow problems. Overflow issues aren’t very common in business situations, numbers there tend to be small compared to the size of a 64 bit signed integer, but in some scientific computation situations and in my Socialist Millionaire implementation encryption overflow is an issue. Sampling from a random number stream and multiplying the results is another source of overflow situations. In fact, random sampling and multiplying has a very high probability of overflow if the random number generator yields results that are close to being evenly distributed.

I want to keep the example in this post simple, so I will show you an implementation of the grade school method. Multiplying 99 x 99 the way I was taught in grade school (multiply, carry, and then add) is a good place to start.


There is an alternative way of doing this same multiplication (multiply, add, then carry). In this example each time 9 is multiplied by 9 the result is written on a new line. All except the first partial product are shifted left —  one place for the second and third product, two places for the fourth product.


The second method is the one I’m going to show in this posting. An example of bits being manipulated is helpful. To make it easier to understand we’ll stick with 8 bit numbers where each nibble (4 bits) represents one ‘digit’ for multiplication purposes.


Following the second grade school multiplication method using nibbles, lets multiply 255 x 255. Multiplying those two numbers should overflow an 8 bit result but we’ll keep that from happening without loosing any data.

Keep in mind that 0b1111 x 0b1111 = 0b11100001 (15 x 15 = 225).
The two part base 256 product, one 8 bit number representing the ones place and the other representing the 256’s place, is the result of left shifting the upper half of two nibble pairs and adding them to their corresponding bottom halves,

1111 << 4 + 1110 yields 11111110 and

0000 << 4 + 0001 yields 00000001.

The code example below follows the same pattern as the 8 bit multiplication without overflow table and uses the constant and variable names listed in the first column.


 * I did this big endian since it was easier to think about this part of the algorithm this way
func multiplyWithOverflow(multiplicand:UInt64, multiplier:UInt64) ->[UInt64]{
    //multiply to get all addends
    let lowerLowBits = (multiplier & bottomHalfBitMask) 
                       * (multiplicand & bottomHalfBitMask)
    let lowerHighBits = (multiplier & bottomHalfBitMask) 
                       * (multiplicand >> 32)
    let upperLowBits = (multiplier >> 32) 
                       * (multiplicand & bottomHalfBitMask)
    let upperHighBits = (multiplier >> 32) 
                       * (multiplicand >> 32)
    //get the 4 bit sums of the addends. There are 4. Some may be zero
    let bottom = lowerLowBits & bottomHalfBitMask
    let middleBottom = (lowerLowBits >> 32) + (lowerHighBits & bottomHalfBitMask) 
                  + (upperLowBits & bottomHalfBitMask) + (bottom >> 32)
    let middleTop = (lowerHighBits >> 32) + (upperLowBits >> 32) 
                  + (upperHighBits & bottomHalfBitMask) + (middleBottom >> 32)
    let top = (upperHighBits >> 32) + (middleTop >> 32)
    //combine the 32 bit sums to make two 64 bit values
    let bottom2N64 = (middleBottom << 32) + bottom
    let top2N64 = (top << 32) + (middleTop & bottomHalfBitMask)
    return [top2N64,bottom2N64]

If you want to use this overflow protection in a situation where overflow is infrequent, business, games, some simulations, etc., you will want to add an if statement to this code or allow the multiplication to overflow and then handle the situation as an error. Checking to see if two numbers will overflow before multiplying them turns out to be fairly easy. All you have to do is find the location of the highest bit in both numbers and add those locations together. If the location sum is greater that 63 the two numbers will overflow when multiplied.

In my next post, I’ll show how to represent huge numbers as arrays of UInt64’s and multiply those together, another component I need when implementing the Socialist Millionaire protocol.

October 14, 2014

Swift, Structs, and Data Storage/Retrieval with sqlite

Posted in Uncategorized at 8:52 pm by tetontech

A data storage and interaction problem exits in Swift. If someone uses structs instead of objects in an attempt to take a more functional approach in coding their application, CoreData won’t work. It only interacts with objects, not structs. While Apple may solve this in the future, that possibility doesn’t help us now. To resolve this issue I decided to create two libraries that would behave like an ORM (Object Relational Mapper) but work with structs (STRUM?).

The first library runs all interactions with sqlite3 on background threads so we don’t have to deal with threading. The sqlite3 interaction is SQL based and generates and uses prepared statements, transactions, and rollbacks. I call this first library SwiftlyDB and its source code is available on my github repository. It is MIT licensed and the main purpose of this posting is discussing its API. Later postings will cover Swift lessons learned during SwiftlyDB’s creation.

The second library, currently in the design stage, will work with structs much like any ORM works with objects. It will translate between the results returned by SwiftlyDB queries and create the appropriate structs. It will also generate the SQL required to store struct information using SwiftlyDB. This means that this second library, which I’m currently calling SwiftlyStore, will make it possible to safely store, update, retrieve, and delete from storage Swift structs in an intelligent and parallel way.

Now let’s get down to how to use SwiftlyDB. SwiftlyDB is functional, parallel, light weight (less than 350 lines of code), flexible, and easy to use. Its API consists of two structs, SwiftlyDb and DBAccessError, and four functions; setupSwiftly, discardSwiftly, swiftlyTransact, and swiftlyTransactAll. The first two functions setup and disconnect from a sqlite3 instance for you and the last two execute SQL.

To get started, use setupSwiftly to generate an instance of SwiftlyDB. You do this by passing it the name of an sqlite file. This example uses ‘test.sqlite.’

let (error,aSwiftlyDB) = setupSwiftly(“test.sqlite”)

If the sqlite file exists in your application’s bundle, setupSwiftly will copy it, only once, to your application’s documents directory so it can be used. If you choose not to ship a database with your app setupSwiftly will create the sqlite file in the documents directory for you. Either way, you end up with a usable sqlite database file for your app.

In the example line of code above, aSwiftlyDB is of type SwiftlyDB optional and error is of type DBAccessError optional. When your file is copied and the sqlite3 database is opened, error will be nil and aSwiftlyDB will not.

‘test.sqlite’ used in this example has only one table, the ‘dog’ table with id:text, age:integer, and height:double fields. To add a single dog use the swiftlyTransact function, swiftlyTransact(aSwiftlyDB:SwiftlyDB, sql:String, parameters:Array<Storable>?, resultHandler:(DBAccessError?, Any?) ->()).

The example below shows how to add a dog without using prepared statements which, by the way, is not usually a good idea. An SQL string is assembled to insert the dog and the string, aSwiftlyDB returned from the call to setupSwiftly, and a closure containing code to execute after the insertion is done are passed in as parameters. As is possible in Swift, the closure is placed after the closing parenthesis to aid in readability.

let insertString = "INSERT INTO dog VALUES (\"\(NSUUID.UUID().UUIDString)\", 5, 3.2)"
swiftlyTransact(aSwiftlyDB!, insertString, nil){(error:DBAccessError?, data:Any?) ->() in
    if let theErrorDescription = error?.description{
        println("oops. got an insertion error. \(theErrorDescription)")
        println("updated \(data!) records")

After the insertion is done using the sqlite3 C library, the result of the insertion operation is passed to the resultHandler closure. In this case, the data parameter is a Int representing how many records were changed (1). I’ve shown some simple error handling made possible by having both an error and the transaction’s data as parameters to the resultHandler.

Now that data is in the database how do we get it out? A second call to swiftlyTransact will do that for us. In the code below, all of the dogs in the table are requested and printed to the console. You could present this to the user via your app’s user interface but I’m trying to keep the example simple.

let selectString = "SELECT * FROM dog"
swiftlyTransact(aSwiftlyDB!, selectString, nil){(error:DBAccessError?, data:Any?) ->() in

    if let theErrorDescription = error?.errorDescription{
        println("oops. got a selection error. \(theErrorDescription)")
    else if let dogs = data as? Array<Dictionary<String,String>>{
        println("found \(dogs.count) in dogs")
        for dog in dogs{
            for fieldName in dog.keys{
                println("\t\(fieldName) : \(dog[fieldName]!)")
       println("done with dogs")

swiftlyTransact also works well for single prepared statements. This example shows how to add a single dog like the previous insert example.

let preparedStatementString = "INSERT INTO dog VALUES (?,?,?)"
swiftlyTransact(aSwiftlyDB!, preparedStatementString, [NSUUID().UUIDString, 5, 3.2]){(error:DBAccessError?, data:Any?) ->() in
    if let theErrorDescription = error?.errorDescription{
        println("oops. got an insertion error. \(theErrorDescription)")
        println("updated \(data!) records")

Any library that forced you to execute a function call for every SQL statement would be poorly designed. SwiftlyDB uses swiftlyTransactAll to fill the roll of executing ‘sql scripts’, swiftlyTransactAll(aSwiftlyDB:SwiftlyDB, tasks:Array<Dictionary<String,Array<Storable>?>>,resultHandler:((DBAccessError?, [Any]?) ->())?) -> ().

swiftlyTransactAll has three parameters, an instance of SwiftlyDB to work with, a list of tasks to do, and a closure to execute once the list of tasks is complete.

This example adds 10 random dogs to the dog table using swiftlyTransactAll.

var taskList = Array<Dictionary<String,Array<Storable>?>>()
for index in 0..<10{
    let parameters:Array<Storable>? = [NSUUID().UUIDString,Int(arc4random()%12),Double(arc4random()%100)/10]
    let aTask = [preparedStatementString:parameters]

swiftlyTransactAll(aSwiftlyDB!,taskList){(error:DBAccessError?, data:[AnyO]?) -> () in
    if let theErrorDescription = error?.description{
        println("oops. got an insertion error. \(theErrorDescription)")
        println("updated \(data!.count) records")

The last function in the API closes the sqlite database for you and cleans up.



So that’s it. The entire API. Most of the code exists in the SwiftlyDB.swift file, but one item of interest exists in the Storable.swift file. It would be good to limit the parameters to types of things that make database sense; Strings, Ints, and Doubles. To this end String, Int, and Double have been extended to implement a custom Storable protocol found in the Storable.swift file. This way types of things that don’t make database sense such as user interface elements, controllers, delegates, etc. can not be accidentally sent to one of the swiftlyTransact functions.

SwiftlyDB is available for your use. It is functional in nature. It handles sqlite database transactions and rollbacks for you. It executes all database interactions in parallel to your code and does this safely. Take it and use it in any way you want.

My next postings will look at Swift lessons learned as I implemented this library.

August 20, 2014

Swift, Extensions, Protocols, and Operator overloading

Posted in Uncategorized tagged , , , , , at 4:39 pm by tetontech

I’m working on a Swift matrix manipulation library. Since I want that library to be flexible enough to be used outside of my planned use, a simulation engine, it needs to be able to handle complex numbers. Swift doesn’t have representations for imaginary or complex numbers so I created one. You can get the source for the entire library at my gitHub repo. I call the library SwiftlyComplex. Regardless of why I created the library, there are some Swift programming concepts the library illustrates.

I wanted my representation of complex numbers to be able to have Int and Double type parts but not String or other invalid types. Swift’s implementation of Int and Double have no common base class so I could have done something silly with Swifts Any type and do a bunch of run-time checking or I could have overloaded all my functions to handle all possible combinations of Int and Double. Both of these are bad ideas. For example, if the complex number’s init function had two parameters, one of the real component and one for the imaginary, using the overloading idea I would need to create four init methods that do essentially the same thing, initialize the complex number’s properties. The overloading approach would also make it difficult to declare the complex number’s property types as well. Not a good approach.

So what I did instead was create a BaseNumeric protocol.

protocol BaseNumeric {
    func asDouble() -> Double
    func asInt() -> Int
extension Int:BaseNumeric {
    func asDouble() -> Double{
        return Double(self)
    func asInt() -> Int{
    return self
extension Double:BaseNumeric {
    func asDouble() -> Double{
        return self
    func asInt() -> Int{
        return Int(self)

As you can see, I used this protocol to extend both Int and Double. This allowed me to make one version of all of the complex number’s init and other functions. I also had BaseNumeric add a couple of helper functions to Int and Double to return Double and Int versions of the number. This reduced code duplication in my complex number representation.

BaseNumeric also needed to override the -, +, *, and / operators. They are all very similar. We’ll look at just the override for the – operator.

func - (lhs:BaseNumeric, rhs:BaseNumeric) -> BaseNumeric{
    if lhs is Double || rhs is Double {
        return lhs.asDouble() - rhs.asDouble()
        return lhs.asInt() - rhs.asInt()

This version of the – operator can be placed between two BaseNumeric values, be they Ints, Doubles, or some combination of those, and returns a BaseNumeric. The operator checks the type to see if either BaseNumeric is a Double. If at least one is, then the result is calculated as a Double. If not, the result is calculated and returned as an Int BaseNumeric.

With the BaseNumeric protocol extending both Int and Double, I could now create a struct to represent complex numbers. I called it Complex. Complex implements Swift’s standard Printable protocol so using println() is easy. Implementing Printable meant I needed to add a calculated property of type String called description.

Following this same pattern, I added two custom calculated properties for the common complex number calculations modulus and conjugate. While you may not be dealing with complex numbers yourself, calculated properties in your code will work the same way. They are declared as variables, the return type is declared, the calculation is done, and a value is returned.

import Foundation

struct Complex:Printable{
    var real:BaseNumeric
    var imaginary:BaseNumeric
    var description: String{
        return "\(self.real) + \(self.imaginary)i"
    var modululus: Double{
        let squaredReal = real * real
        let squaredImaginary = imaginary * imaginary
        return sqrt((squaredImaginary + squaredReal).asDouble())
    var conjugate: Complex{
        let inversImaginary = imaginary * -1
        return Complex(real: real, imaginary: inversImaginary)
    func combine(rhs:Complex, combineBehavior:(BaseNumeric, BaseNumeric) -> BaseNumeric) -> Complex{
        var realPart = combineBehavior(self.real, rhs.real)
        var imaginaryPart = combineBehavior(self.imaginary, rhs.imaginary)
        return Complex(real: realPart, imaginary: imaginaryPart)

I also added a combine function. This function allowed me to do both addition and subtraction of my Complex structs with one set of code rather than duplicate the code for both behaviors. To accomplish this, I needed to pass an instance of a Complex to combine with the ‘self’ instance, and a closure, called combineBehavior in the parameter list. The closure would either add or subtract the component pieces of the Complex structs depending on if I wanted addition or subtraction. Calling the combine function in an overloaded version of the + operator shows how to pass the closure.

func +(lhs: Complex, rhs: Complex) -> Complex{
    return lhs.combine(rhs, combineBehavior: {(leftValue:BaseNumeric,rightValue:BaseNumeric) -> BaseNumeric in
        return leftValue + rightValue

Since + is defined by Swift as an infix operator, I didn’t need to declare it myself. For a discussion of overloading standard operators and creating custom operators please see my previous posting on that topic.

In addition to the + and – operators, I overloaded the  * and / operators for my Complex class. They calculate the multiplication and division of complex numbers directly since there is not common code between these operators that could be shared.

Now I can create complex numbers using the Complex structure.

     let first = Complex(real: -2, imaginary: 3)
     let second = Complex(real: 1.0, imaginary: 2.0)

and manipulate them using the overloaded operators.

     let difference = first - second
     println("difference: \(difference)")
     let product = first * second
     println("product: \(product)")
     let quotient = first / second
     println("quotient: \(quotient)")

This makes it much easier to create, manipulate, and display complex numbers.

July 15, 2014

Swift to JavaScript and JavaScript to Swift. A round trip.

Posted in Uncategorized tagged , , , , , at 9:12 pm by tetontech

Edit: Since originally writing this post I have Created the SwiftlyHybrid and AndyHybrid libraries and put them on gitHub. If you are interested in the topic discussed here I would suggest also reading the SwiftlyHybrid and AndyHybrid follow up posting.

In a previous post I showed how to make a call from JavaScript to Swift. Take a look there for a more in-depth discussion of the code to accomplish that data transfer. This example expands on what I did in that post by showing how to make a call from JavaScript to Swift, do a calculation, and then pass the results back to JavaScript. The source code is very similar. It uses one method, evaluateJavaScript(), of the WKWebView class that wasn’t available in Apple’s previous Swift beta.

In the view controller’s userContentController method we’ll need to have access to the WKWebView created in the viewDidLoad method. This requires the addition of a WKWebView attribute to the ViewController class. After trying a couple of experimental ideas I remembered the correct, Swifty way to do this. I couldn’t create the WebView on the same line as the attribute was declared since the initializer for the WKWebView isn’t as simple as adding some integer attribute (var someInt = 3). Instead the initializer needs a frame and a WKWebViewConfiguration. Because of this the WKWebView must be initialized in the viewDidLoad method (This may change if Apple adds a WKWebView to Interface Builder).

The Swift way to declare an attribute and initialize it in something other than an init method is to make the attribute optional using the ? operator.

class ViewController: UIViewController, WKScriptMessageHandler {
    var theWebView: WKWebView?

In the viewDidLoad method the WKWebView can now be created using the WKWebView function. Once theWebView has been set, the ! operator is needed to declare that we know theWebView attribute is not nil.

(Warning: the functionallity of WKWebView has changed since this code was written. You must move all resources (html, JavaScript, Css, images, and other files) into the temp directory or WKWebView will not load the resources. This may change in the future as WebKit changes or it might not.)

    override func viewDidLoad() {
        var path = NSBundle.mainBundle().pathForResource("index", 
                               ofType: "html")
        var url = NSURL(fileURLWithPath: path)
        var request = NSURLRequest(URL: url)

        var theConfiguration = WKWebViewConfiguration()
                                   name: "interOp")

        theWebView = WKWebView(frame: self.view.frame, 
                             configuration: theConfiguration)


As in the previous example, the userContentController() method must be implemented to handle the messages captured by the script message handler that was added to theConfiguration. In this example the message’s body, the data sent from the JavaScript is interpreted as an NSDictionary since the JavaScript sends an associative array.

    func userContentController(userContentController: 
       didReceiveScriptMessage message: WKScriptMessage!) {
        let sentData = message.body as NSDictionary
        let aCount:Int = Int(sentData["count"] as NSNumber)
        theWebView!.evaluateJavaScript("storeAndShow( \(aCount + 1) )", 
                                        completionHandler: nil)

The other change to the previous example is the addition of the call to theWebView’s evaluateJavaScript method. It has two parameters; a string that is the Javascript to be executed, and a function or closure that will be notified once the JavaScript completes. In this example I don’t need to know when the JavaScript completes so I passed nil as the second parameter.

The javascript for this example is strait forward; an onclick listener function to send a message to the Swift message handler, and the storeAndShow function that the Swift code will call to send a message back to the JavaScript.

var count = 0

function sendCount(){
    var message = {"count":count}

function storeAndShow(updatedCount){
    count = updatedCount
    document.querySelector("#resultDisplay").innerHTML = count

For my own code I would send a Dictionary back to the JavaScript where it would be used as an associative array. I left that out of this example since I didn’t think it was necessary to show.

Overall, the interaction between Swift and Javascript using the WKWebView is more strait forward than using the old UIWebView. It is nice to have a JavaScript way to directly call to Swift. I also like the selection of JSON as the data transfer format for the JS to Swift communication. It reflects the decision I made to adopt that format for QCHybrid years ago. It would be better if the values in the dictionary knew their type so casting using the as operator wasn’t needed but I can live with it the way it is.

Next page


Get every new post delivered to your Inbox.

Join 329 other followers

%d bloggers like this: