Luminescent Dreams

FitnessTrax 0.1.0

March 27, 2020
The Streets at Night, 2019-09

Hello, and welcome back!

Today, I get to announce that I have release FitnessTrax 0.1!

FitnessTrax is a health and workout tracking program designed for individual users and following the privacy policy of keeping all of your data on your own computer. Data that lives there cannot be mined and exploited by third party companies that may not really have your privacy in mind.

This is the first version of the application that I feel is suitable for home users. While it is rather ugly, and limited at this time, it is also entirely usable. Over the next few releases, I plan to style the application nicely and to add many other pieces of information to the tracking system.

FitnessTrax is a GTK 3 application. It can currently only be installed from source. A NixOS derivation will be available soon. I am not currently working on installation from Crates.IO, on Ubuntu, or with Flatpack, but all three of these are on my roadmap.

History

This is the culmination of many years of work and learning.

I first started tracking my weight and my bike rides in 2003. I used nothing more than a couple of plain text files. For a long time, that worked, but graphs were impossible, and it was not possible for me to get a coherent view of any particular day in my history without reading information from many different places.

In 2013, I finally started trying to put all of the information into a single database of sorts. From the beginning, I had no desire to set up a separate database server. Aside from simply being overkill for an app that will only recieve a few writes every day, a separate server increases the effort necessary to install an application, and makes data backups significantly harder.

So, I started with a SQLite database and a Haskell web service. Over the years I migrated from SQLite to building an embedded time series database, then from Haskell to Haskell and Javascript, then to Rust and Javascript, Rust and Typescript, and finally to a Rust-only native GTK application.

Over the years I modified, updated, and tweaked the various data models into something that mostly fit the patterns that I could understand. I settled, at long last, on the Record data structure, then defined Weight, Steps, TimeDistance, Duration, and SetRep as records. Further, in my time at Cloud City I learned from several people who are very good at architecting user interfaces, and that has lead to the internal architecture of my application.

It may seem silly that an application that is little more than a glorified spreadsheet could take so long to build. But it should be known that I have developed this application mostly on my own, and have taken many false steps in the process. I have learned a lot, most of which I hope to one day apply to my career. I do not think that I could have ever done any better.

Honoring Amelia, two years later

January 31, 2020

In Memory of Amelia Perry

Two years ago today, I woke up to the news that my friend, Amelia, was gone.

Amelia was the victim of our country’s transphobia and inadequate response to mental health needs. She ended her life after six months intermittently in hospitals. Since then, I have participated in some of the same hospitalization programs as her. While I have been able to recover, her experiences continue to remind me htat these systems are inadequate, and that transgender inclusion cannot be assumed even in a state whose laws support transgender rights.

My feelings are complicated. I do not feel acute pain any more, but I believe that is because I remember her, and feel a bit of pain, every single day. Every day I think about the things she could be doing, and the things she already did to advance math and science. Things that I am unable to understand because she was beyond brilliant.

So, today, we remember the anniversary of Amelia’s death and we continue to fight for the inclusion and the dignity of transgender people, and destigmatization of mental illness until it is as unremarkable as any other illness.

Rust and GTK from a React perspective

January 15, 2020

Recently, after a few failed attempts at using other frameworks to make an application that was both easy to use and easy to install, I embraced native software development with Rust and GTK.

Though I have made short forays in the past, GTK was a change for me. Before this, most of my user interface experience came from building React applications. The transition from React to GTK posed some challenges. Most came from differences in widget philosophy. GTK in Rust, though, is particularly hard because of the extra rules Rust enforces to protect against memory management errors and against operations that are unsafe to do in a threaded context.

In this article, I will talk primarily how I adapted the philosophies from React into GTK, and I will highlight some of the extra tricks that are necessary to make GTK conform to Rust’s rules. Rust enforces some tricky rules that will be unfamiliar to most developers, primarily in terms of how values can be shared, but also with strong restrictions on mutability. I’ll point out these rules as they come up throughout this article.

All of the examples in this article come from FitnessTrax, a privacy-first fitness tracking application. Users are able to collect fitness and biometric data in a single place on their personal computers, without depending on companies that may not appropriately protect their user’s data in the long term.

I apologize for the appearance of the application, because as of the 0.4 release I have not taken the time to learn much about how GTK handles styling. I promise that I will improve the UI significantly soon.

Some differences in framework philosophy

The developer of Conrod, a graphical toolkit for Rust that experiments with applying Functional Reactive Programming techniques to graphics programming, describes two significantly different modes for managing graphical components. In “Retained Mode”, which is the common mode for most native graphical programming, any given screen component is created, then updated repeatedly throughout its lifetime. In “Immediate Mode”, components will have a draw method in which they freshly instantiate all of their children. The framework then compares this tree to the previous tree to determine how to update the screen.

React operates entirely in Immediate mode, while GTK operates entirely in Retained mode. In web developement, D3, a popular data visualization library, also works in retained mode, and in 2018 I wrote an article about interfacing between React and D3.

React paired with Redux or Apollo-GraphQL implements some of the concepts of Functional Reactive Programming in that it automatically handles propogating data changes to components. My introduction to FRP came from Elise Huard’s book “Game Programming in Haskell”. This book may be getting out of date by this point, but it does serve as a good introduction to the concept in the context of a particular FRP library in Haskell. Unfortunately, FRP has not seen wide adoption outside of React. Though there is at least one FRP library available for Rust, at the time of this writing it feels a bit too immature for me to adopt. As such, with a bit of creativity and my experience with React, I have designed mechanisms that approximate the FRP paradigm.

A note on terminology:

  • A widget is a GTK object which represents something on screen. This could be a Window, a Button, a Label, or a Layout container. GTK widgets can only have other GTK widgets as their children.
  • A component is any logical abstraction of a section of the screen. In simple cases, this will be a GTK widget returned from a function. In more complex cases, it may be a structure which contains one or more widgets. Components cannot necessarily be passed into GTK functions. Structure components always provide a public widget field which represents the root widget for this component.

An immutable value display

The simplest of all components is, much like a React component, a small collection of widgets that gets created and then never gets updated. This can be implemented simply as a function that returns a GTK widget.

A simple display showing the date January 13, 2020

pub fn date_c(date: &chrono::Date<chrono_tz::Tz>) -> gtk::Label {
    gtk::Label::new(Some(&format!("{}", date.format("%B %e, %Y"))))
}

This pattern works when a component really is meant to be a visual component that rarely, or even never, updates. In my application, date labels are subcomponents of larger displays, and thus are the kind of thing that never change.

A component with internal widget state

Components with internal widget state only can be significantly more complex, yet can still be implemented as a function which returns a GTK widget. The the caller could read the data directly out of the returned GTK widget, this pattern arguably works best when the caller supplies a callback, and the component encodes rules for when to call the callback.

I have a validated text entry field. It is a regular gtk::Entry, but the interface abstracts the text handling behind render, parse, and on_update functions.

Two text entry fields. The one on the left shows an orange border to indicate that the value “15.abc” is not valid as a distance. The right one has a black border to indicate that “1:15:00” is a valid duration.

pub fn validated_text_entry_c<A: 'static + Clone>(
    value: A,
    render: Box<dyn Fn(&A) -> String>,
    parse: Box<dyn Fn(&str) -> Result<A, Error>>,
    on_update: Box<dyn Fn(A)>,
) -> gtk::Entry {
    let widget = gtk::Entry::new();
    widget.set_text(&render(&value));

    let w = widget.clone();
    widget.connect_changed(move |v| match v.get_text() {
        Some(ref s) => match parse(s.as_str()) {
            ...
        },
        None => (),
    });

    widget
}

The caller must provide an initial value, a render function, a parse function, and an on_update function. In my implementation, the validated text entry will attempt to parse the string within the box after each change, and will call the on_update function only if parsing succeeds. The caller is thus responsible for saving the data, but does not have to worry about the mechanics around parsing or verifying that it has valid data.

I find this pattern particularly useful in forms where I opt to store all of the values of a form together in one place. Storing all of the data together lets me notify the user immediately of errors, lets me detect errors that occur as a result of invalid combinations of data, and lets me easily disable the Submit button when there are errors present.

Components with internal state

2020-01-31: it turns out that I have made some big mistakes in the code in this section. I will need to revise it pretty significantly to handle more efficient component updates, and changing component state in a GTK callback.

While I build my application out of simple components like those above, I put them together into more sophisticated components that have multiple pieces of data that logically belong together but mechanically get edited within the various subcomponents. For this, I set up an internal state independently of the state of the subcomponents.

Fortunately, I can still usually implement this as a function.

Take the instance of a bike ride, which I have abstracted to a “time/distance” record. A time/distance event has a start time, an activity type (bike ride, walk, run, kayak trip…), a distance, and a duration. My user interface binds all of these together into a single component that worcks on the entire record at once.

An edit interface for information about a bike ride. It is a single row containing the time of day that the ride started, a dropdown menu in which Cycling is currently selected, a distance of 15.5km, and a duration of 1 hour, 15 minutes.

pub time_distance_record_edit_c(
    record: TimeDistanceRecord,
    on_update: Box<dyn Fn(TimeDistanceRecord)>,
    ) -> gtk::Box {
}

Here we start to run into the rules that Rust enforces to guarantee safe memory management.

Every value has exactly one owner. While you can get a borrowed reference to that value, those references must go out of scope before that value’s owner goes out of scope. Additionally, you can only get a mutable reference if there are no other references of any kind. The Rust Book talks in detail about these rules and provides a significant number of examples and scenarios.

Fortunately, all of the parts are already here. I need a way to share the record across multiple callback functions, and I need a way to ensure safe multithreaded access to the record. We solve the sharing problem with an Arc. This is a thread-safe reference counted container. Any value passed to the Arc’s initializer becomes owned by the Arc. Cloning an Arc increments the reference count and creates a second reference that points to the shared value.

Arcs do not allow mutable access to the values they contain, so we need to also include an RwLock. As expected, an RwLock allows many readers but only a single writer, and no readers are allowed when there is a writer. So, here is how we safely mutate the record:

pub time_distance_record_edit_c(
    record: TimeDistanceRecord,
    ...) -> gtk::Box {

    let record_ref = Arc::new(RwLock::new(record));

    {
        let mut rec = record_ref.write().unwrap();
        ref.activity = Cycling
    }

Within the sub-block of code, rec becomes a mutable reference to the record data. RwLock governs read/write access to the data, while Arc allows the data to be shared across functions or even threads.

Putting it all together, our code looks like this:

pub time_distance_record_edit_c(
    record: TimeDistanceRecord,
    ...
    on_update: Box<dyn Fn(TimeDistanceRecord)>,
) -> gtk::Box {

    let on_update = Arc::new(on_update);
    let record = Arc::new(RwLock::new(record));

    let duration_entry = {
        let record = record.clone();
        let on_update = on_update.clone();
        let duration = record.read().unwrap().duration.clone();
        duration_edit_c(
            &duration,
            Box::new(move |res| match res {
                Some(val) => {
                    let mut r = record.write().unwrap();
                    r.duration = Some(val);
                    on_update(r.clone());
                }
                None => (),
            }),
        )
    };
}

(note: functions are always read-only and so require only the Arc for sharing)

To recap, in the above function, we have a block of code which clones the Arc containing the record. That clone gets moved into the callback function for duration_edit_c (meaning that the callback function now owns that particular clone). Within the callback funuction, the record will be borrowed mutably, updated, the data cloned and passed to on_update, and then the write lock will be automatically dropped at the end of the block.

This is a lot to absorb all at once. If you are not familiar with Rust, I definitely recommend reading about the ownership and borrow system, which is the magic that takes memory management away from the developer without incurring the cost of a garbage collector.

Updating from system state changes

Finally, the fourth pattern covers all components that need to respond to system changes. In React terms, this means property changes, possibly from Redux.

At a high level, we need a struct which keeps track of all of the visual components that may be updated given new data, and a render function which will handle those updates and return the root level widget.

For this example, I provide my History component.

An image illustrating the main health history interface. The left side contains two buttons, one atop the other, labelled with January 6, 2020, and January 13, 2020. To the right side are day entrcies for January 13, 2020 and January 12, 2020, with a variety of components within each.
struct HistoryComponent {
    widget: gtk::Box,
    history_box: gtk::Box,
}

pub struct History {
    component: Option<HistoryComponent>,
    ctx: Arc<RwLock<AppContext>>,
}

impl History {
    pub fn new(ctx: Arc<RwLock<AppContext>>) -> History { ... }

    pub fn render(
        &mut self,
        range: DateRange,
        records: Vec<Record<TraxRecord>>,
    ) -> &gtk::Box { ... }

The constructor here is actually quite simple, doing nothing more than creating the abstract History component. It doesn’t even create the widget at this point, as it has no data to populate into the widget. This is rather convenient because at construction time components may require data that is not available yet.

The bulk of the work appears in render:

    pub fn render(
        &mut self,
        range: DateRange,
        records: Vec<Record<TraxRecord>>,
    ) -> &gtk::Box {
        match self.component {
            None => {
                let widget = gtk::Box::new(gtk::Orientation::Horizontal, 5);

                /* create and show all of the widgets */

                self.component = Some(HistoryComponent {
                    widget,
                    history_box,
                });

                self.render(prefs, range, records)
            }
            Some(HistoryComponent {...}) => {
                ..
            }
        }
    }

If this is the first call to render, the visual components will not exist yet. Render will create all of the components, and then call itself again to populate them with the data.

    pub fn render(
        &mut self,
        range: DateRange,
        records: Vec<Record<TraxRecord>>,
    ) -> &gtk::Box {
        match self.component {
            None => {
                ...
            }
            Some(HistoryComponent {
                ref widget,
                ref history_box,
                ...
            }) => {
                history_box.foreach(|child| child.destroy());
                records.iter().for_each(|record| {
                    let ctx = self.ctx.clone();
                    let day = Day::new(
                        record.clone(),
                        ctx,
                    );
                    day.show();
                    history_box.pack_start(&day.widget, true, true, 25);
                });
                &widget
            }
        }
    }

On subsequent calls, render will handle updating the widgets. The details of how to populate the new data will vary pretty significantly by component. In this case I destroy all of the existing subcomponents and create new ones based on the data that I have. This is a pretty naive strategy, but sometimes it works.

Conclusion

And so, here they are. Four high level patterns that I discovered through weeks of learning how to program GTK. I doubt that this will be their final form.

Even over the course of writing this article I significantly modified, refactored, and simplified my components. I imagine that these four patterns will take me very far in this application, while also expecting to learn much more as I proceed.

A bit of freehand sketching

January 14, 2020
A very rough sketch featuring several plain squares with some rough perspective. The sketch represents my living room. There is a dark square in the center representing my TV, with multiple arches representing the various openings in the wall.

As a change of pace, today I’ll share a rough freehand sketch that I made this morning. I was sitting in my favorite spot in my living room and decided to do a quick fifteen minute “speed sketch”. This one depicts my TV, the stand it sits atop, and the various doorways leading out of the room. It basically skips out on all of the details sitting around.

I was good at sketching back in high school when I took a semester-long class of it, but I’ve largely stopped doing it. However, my partner has decided to start sketching as a way of driving her artwork, and I just added wireless capability to my wacom tablet, so I’ve suddenly decided to do some of the same. Though perhaps not at the one-a-day rate that she’s going for.

Art is something I’ve been neglecting, and I note that I don’t think I have taken one single photograph with my good camera yet this year.

So, in penance, I will share a phone photo of Sapphira looking all comfortable.

A white cat with sapphire-blue eyes and black ears. She is looking at the camera with only her head visible as the rest of her body is underneath a red comfortor. She appears very comfortable.

Fitnesstrax 0.0.4 and GTK

January 03, 2020
_DSC3706.web

I finally return after a bunch of weeks. December was an incredibly hard month, and the mental health problems I referenced in my last post actually became significantly more pronounced. I found that I had to take a large amount of time away from the world in order to get things back together.

Last time I got this bad, back in 2013, I had the freedom to quit my job, sell my house, and go live in the woods for a year. Not so this time, as I still haven’t recovered from the financial impact of trying to control my re-entry into the workforce.

Nevertheless, I am probably better prepared, and better supported, that I was back in 2013.

FitnessTrax

I am pleased to finally do a new release of FitnessTrax!

This represents a significant technology shift. Instead of a web client/server application, this is now a dedicated GTK application. This should make it significantly easier for regular users, though right now I am only building and testing on Linux.

However, forward progress also means some backsliding. Here’s some differences from the previous web app:

  • You need a configuration file – fortunately, easy to set up
  • Most configuration is unsupported – you are stuck in English with the metric system, but timezones work
  • Weight, step counting, and time/distance workouts
  • No Set/rep workouts or duration-only workouts

I hope to move forward quickly on all of these things, especially now that I have largely discovered patterns that allow me to work with Rust and GTK together. Version 0.0.5 will have much better configuration support, including the ability to reconfigure and have the configuration options automatically saved.

So, to get the application, you can either get it from my Nix chcannel, or you can clone the repository and build it yourself.

FitnessTrax in Nix

This is certainly the easiest way to install FitnessTrax. Note, however, that it is most likely to work on various Linux distributions all running the Nix package manager. It may or may not work on MacOS.

nix-channel --add http://luminescent-dreams-apps.s3-website-us-west-2.amazonaws.com/nixexprs.tar.bz2 luminescent-dreams
nix-channel --update

nix-env -i -A luminescent-dreams.fitnesstrax_0_0_4

FitnessTrax from Source

If you want to build from source, start by cloning the repository:

git clone https://github.com/luminescent-dreams/fitnesstrax.git

You will need to install a variety of GTK development libraries, including Pango, Cairo, and possibly also Atk. Refer to your package manager for more information. You will also need Rust 1.39 (or newer, but I’ve built and tested on 1.39).

Once you have the libraries, you can build the GTK aplication:

cd fitnesstrax/gtk
cargo build --release

You may need to add version numbers to the two relevant Cargo files, as I removed the version numbers on the assumption that all of my distribution would be managed through Nix. I recommend version 0.0.4 for both, and I will fix that problem when I release 0.0.5.

Configuration file and starting the application

Once you have your executable, you’ll need to create a configuration file and then tell FitnessTrax where to find it. You can put it anywhere, but I rather like ~/.fitnesstrax.yaml and may make that the default location some time in the future.

---
series_path: /home/savanni/health/health.series
timezone: America/New_York
language: en

All three fields are required, even though not all three are supported.

Important note about timezone: it must be in the format specified in the Olsen Timezone Database. This will be more familiar than you expect as it is the same list of timezones available when you configure a Mac or an Ubuntu system.

Now, on the command line, this should start the application (replace the configuration file with the path to your configuration file):

CONFIG=~/.fitnesstrax.yaml fitnesstrax

The actual name of the configuration variable will change in version 0.0.5, as will how the configuration file gets handled.

GTK and Rust

Rust has a largely complete GTK binding. However, there are not many good examples of how to build a multi-threaded application with it. GTK is inherently single-threaded, and the Rust bindings enforce that by not providing a Send implementation for any of the widgets. This makes it impossible to move any of the widgets, or even references to those widgets, into secondary threads.

While the team is working on Async/Await support for the bindings, I have no idea what form those will take.

In order to do a multi-threaded application, they have provided a channel that receives messages on the main GTK loop. As normal, you receive both the transmit and receive channels. You must then attach a callback function which will be executed for every message that arrives on the receive channel:

    application.connect_activate(move |app| {
        let (tx, rx) = glib::MainContext::channel(glib::PRIORITY_DEFAULT);

        let ctx = Arc::new(RwLock::new(context::AppContext::new(tx).unwrap()));
        let gui = Arc::new(RwLock::new(components::MainWindow::new(ctx.clone(), app)));

        let gui_clone = gui.clone();
        rx.attach(None, move |msg| {
            gui_clone.write().unwrap().update_from(msg);
            glib::Continue(true)
        });

After this, I have modelled the components that I write pretty heavily on my experience with React. I create a lot of composite components, and as much as I can I try to make them stateless. For instance:

pub fn weight_record_c(record: &WeightRecord) -> gtk::Label {
    gtk::Label::new(Some(&format!("{} kg", &record.weight.value_unsafe)))
}

Or, for something more sophisticated, I may build a struct:

#[derive(Clone)]
pub struct ValidatedTextEntry<A: Clone> {
    pub widget: gtk::Entry,
    value: Arc<RwLock<Result<A, Error>>>,
}

impl<A: Clone> ValidatedTextEntry<A> {
    pub fn new(
        value: A,
        render: Box<dyn Fn(&A) -> String>,
        parse: Box<dyn Fn(&str) -> Result<A, Error>>,
        on_update: Box<dyn Fn(A)>,
    ) -> ValidatedTextEntry<A>
    where
        A: 'static + Clone,
    {
        ...
    }

In this case, the constructor is enough as all of the values get wired together inside the constructor.

This is not React, and there are no declarative automatic by-property updates. So I have to code those myself. This requires some clever thinking about coding. Most components can simply be updated in response to things that are happening internally. For instance, the ValidatedTextEntry above has multiple behaviors attached that respond to how the field changes.

The trickier work involves application-level changes, such as saving data to disk or changing the range of data that should be visible. Aside from the message handler above, some components have an update_from method so that they can update in response to global application events. For instance:

pub struct History {
    pub widget: gtk::Box,
    range_bar: RangeSelector,
    scrolling_history: gtk::ScrolledWindow,
    history_box: gtk::Box,
    ctx: Arc<RwLock<AppContext>>,
}

impl History {
    ...
    pub fn update_from(&self, range: DateRange, history: Vec<Record<TraxRecord>>) {
        ...
    }

}

History is the dominant view of my application, showing every day of information. In response to a ChangeRange{ DateRange, Vec<Record<TraxRecord>> } or a RecordsUpdated{ DateRange, Vec<Record<TraxRecord>> } message, it will delete all of the visible components and create new ones based on the data available. Remarkably, this happens with no visible flicker and is basically instantaneous on the data that most humans will want to look at.

This provides only an overview of GTK + Rust programming. There is a lot more for me to talk about, and I think in future weeks I will take a more detailed dive into the various component architectures that I create, as well as the overall application architecture. For now, consider this a simple teaser. However, this is an open source application, so feel free to take a look at more things in the application.

Looking to Fitnesstrax 0.0.5

For the next version, which I hope will only take a few weeks, I’m going to get configuration working. This means supporting the configuration UI, plus having the rest of the UI translate based on timezone, language, and measurement system.