Flutter, an Ideal Framework for Algorithmic Art?

Flutter is a cross-platform framework for apps. On most of its target platforms, it uses a low-level renderer called Skia to draw user interfaces, rather than wrapping the native platform's UI primitives. This means that you can go very deep with just a single language/context. This also seems to allow for much easier porting to a new platform, as more of the code is portable. Indeed, Flutter seems to be in a much stronger position on macOS and Linux compared to React Native. The web is also a target. But more on all of this later.

It uses a language called Dart, which can run in two modes. The first is an interpreted mode for hot stateful reloading, so there is a great developer experience; you make a change and see it almost immediately without having to rebuild your app. If you are used to web development, this may not seem impressive, but native mobile apps can often take (literally) minutes to compile. The second mode is a compiled one for production (so at runtime, the code is 'native'). Dart is very easy to learn, especially with the excellent tooling that Dart and Flutter ship.

A lot of these features are of massive importance to creative coding. Being able to quickly make changes and see the effect is essential for iterating on artwork. Being able to deploy to a desktop platform for an exhibit, mobile apps for engagement, and the web for promotion would be ideal. Performance is something we don't want to hold us back.

There are some gaps to fill in, but Flutter offers a remarkably good basis for creating 2D, interactive, or procedural work. It is easy, friendly, fast, easy to change and can be delivered to many platforms and contexts.

Out of the box experience for Algorithmic Art

In Flutter, to do fully custom drawing, we subclass CustomPainter and implement a paint method:

  @override
  void paint(Canvas canvas, Size size) {

  }

This Canvas has the broad range of standard drawing functionalities you might expect, for example, drawImage, drawRect, and drawVertices.

Canvas APIs make a lot of use of Paint to control the way drawing happens. Here, we can control things like fill and stroke, blend mode, antialiasing, and more. We can also set a shader which allows for gradients or image drawing.

We include this in an app with a CustomPaint widget (in Flutter, we call components Widgets). The size is controlled either by a child widget (useful for when it is dynamic) or can be specified explicitly.

Capturing Images

One interesting feature that is quite straightforward to add is capturing parts of your UI as images. For any kind of algorithmic art application, this is great, as we'd often want to provide some way to export creations. It requires a bit of setup with a GlobalKey to identify the thing you are interested in, but the key parts look like this:

  static const platform = MethodChannel("your channel name");
  GlobalKey globalKey = GlobalKey();

  Future<void> _capturePng() async {
    final RenderRepaintBoundary boundary = globalKey.currentContext!.findRenderObject()! as RenderRepaintBoundary;
    final ui.Image image = await boundary.toImage(pixelRatio: 4.0);
    final ByteData? byteData = await image.toByteData(format: ui.ImageByteFormat.png);
    final Uint8List pngBytes = byteData!.buffer.asUint8List();
    await platform.invokeMethod("savePng", pngBytes);
  }

// And in your build method (the part that creates the UI)
  @override
  Widget build(BuildContext context) {
    return RepaintBoundary(
      key: globalKey,
      child: // stuff to snapshot
    );

Notice how the pixelRatio allows us to control the quality. This is assuming that a MethodChannel has been set up to allow for the platform to save the image (i.e., a platform-specific implementation), though you could likely avoid this by using a package to do it for you. So, we can easily do fully custom drawing and (relatively) easily export these drawings.

Standard and fancier (mobile) UI

We can do various other things such as simple animations with TweenAnimationBuilder or we can do more interactive experiences based on taps or gestures for mobile devices.

Here is a very simple example of a dropdown menu. Unlike many creative coding frameworks, this (and a lot more) is very easy to create from built-in components.

And here is a simple example of adding sliders to control parameters.

We also benefit from Flutter's massive reach in terms of platforms, good performance, and hot stateful reload, which allows us to quickly iterate. In some ways, this is already ahead of things like Processing/P5.js. Though there are some gaps to fill.

Filling in the gaps and tips on using Dart

How can we improve the AX (Artist Experience)? Flutter gives us a lot of great low level tools for drawing. Indeed, out of the box, we are not too far off the experience of something like Processing. We also have a bunch of great higher-level components for UI, such as buttons, form fields, layouts, and way, way more, as this is the main intention of Flutter.

I tackled the same problem for the 2D Canvas web API with Solandra, so in running into this challenge, I decided rather than just trying to solve for a single project, I should create a library. Hmmmm. At least having created a similar thing before I felt like I had a very good understanding of the goals. And Dart (plus Flutter) actually has a standard library and a bunch of built-in features for things like 2D math, so it should be a lot easier this time?

For the first version I wanted to tackle the following:

  • Better Color: Flutter only(?) supports RGB(A); this is terrible for generative uses. It is okay if you are just copying something a designer sent.
  • Canvas-aware iteration: something from Solandra; basically, I should be able to iterate over the canvas in intuitive ways without having to write a bunch of repetitive boilerplate.
  • A higher-level shape/Path class. Flutter supports a basic Path, but I can't do anything other than describe something to be drawn with it.
  • Pseudo-randomness: I'd want a simple, consistently seeded set of methods for common distributions.

I also wanted to make things concise where possible and always allow for dropping down to the lower Canvas/Paint level.

Colour

I created a helper to create a Color from hue, saturation, lightness, and (typically optionally exposed) alpha.

Color fromHSLA(double h, double s, double l, double a)

This seems to me to be clearly the best of the commonly used ways of constructing colors. As is fairly commonly the case, the hue goes from 0 to 360 (as in degrees); the other values go from 0 to 100. This seems to me like a good set of choices that are quite easy to think about intuitively.

This is typically exposed via APIs like (In Dart, you can have optional positional parameters in square brackets like this or named parameters, which are optional by default):

background(double h, double s, double l, [double a = 100]) {

Solandra has helpers like setFillColor that allow you to set a fill colour to be used when filling Paths. (This is just updating a Paint object, which you can use directly when required for more specific things.)

Iteration

In the original Solandra, I added a bunch of methods to iterate over a canvas. This can make it much easier to start drawing something and allow you to focus on your work. So you can easily create something like:

Here is the Flutter version of forTiling (there are a bunch of similar methods available):

  void forTiling(
      {required int n,
      bool square = true,
      double margin = 0,
      bool columnFirst = true,
      required Function(Area area) callback}) {

In the original TypeScript version I used a callback with 4 arguments. In TypeScript, function parameters can be substituted for something more general—a function with fewer arguments—because of how arguments work in JavaScript. In Dart, this isn't possible, so I added an Area class that gives the useful information for these kinds of functions to work with (notice the this. shorthand that means we don't have to write out so much constructor boilerplate).

class Area {
  Point<double> origin;
  Size delta;
  Point<double> center;
  int index;

  Area(this.origin, this.delta, this.center, this.index);
}

So in action this might look like:

final sol = Solandra(canvas, size);
sol.clipped();
sol.background(40, 20, 90);
sol.forTiling(
    n: 32,
    margin: 0.05,
    columnFirst: columnFirst,
    callback: (area) {
        sol.setFillColor(area.index.toDouble(), 70, 40);
        sol.fill(SPath.rect(at: area.origin, size: area.delta));
    });

Even this simple example hopefully shows how a bunch of things, like adding a margin or tiling over a canvas, become very simple. You can also see how it borrows some ideas from Processing (having a fill color, set via setFillColor, and the ease of the background fill).

Shapes

In Flutter we can easily draw a specific shape. But what if we want to work with a shape by smoothing it out, exploding it into fragments, and so on? For this I created SPath. Let's jump straight into an example. What if I wanted to take a 20-sided regular polygon, split it into triangular segments, and explode those segments into triangular shards a couple of times; now take all those shapes and colour them based on index? Before I show you 'the' answer, first consider what that would be like to implement from scratch. Imagine how fiddly, difficult, and error-prone it would be.

In Solandra this is just:

SPath.regularPolygon(radius: size.minimum * 0.4, at: sol.center, n: 20)
  .segmented
  .expand((p) => p.exploded(scale: 0.75, magnitude: 1.1))
  .expand((p) => p.exploded(scale: 0.75, magnitude: 1.3))
  .forEachIndexed((i, p) {
    sol.setFillColor(i * 5, 80, 70);
    sol.fill(p);
  });

This is both concise and agile (easy to change details or even its structure). (expand is the Dart version of flatMap; i.e., it takes a List, a function from an item of the List to a new List, and concatenates the results.) One other nice operation you can perform on an SPath is the Chaikin algorithm for smoothing out shapes: this cuts each corner to smooth things out nicely. Just a few iterations will produce something that looks nicely curved.

Pseudo Randomness

Solandra supports a bunch of standard pseudo-random operations. randomPoint is interesting, as it takes into account the canvas size.

Point<double> randomPoint()
double randomAngle()
double random()
int randomInt(int max)
bool randomBool()
double gaussian({double mean = 0, double sd = 1})
int poisson(int lambda)
T sample<T>(List<T> items)
List<T> samples<T>(List<T> items, int count)
doProportion(double proportion, Function() callback)
proportionately(List<Case> cases)
Point<double> perturb(
    {required Point<double> at, required double magnitude})
List<T> shuffle<T>(List<T> items)

The use of seed is consistent, and I think this covers most common distributions. It also includes helpers for randomized control flow, such as doProportion (do a thing this proportion of times) and proportionately when there are a set of cases to pick from randomly.

Helpers

I added some basic canvas helpers:

  clipped()
  double get aspectRatio
  double get width
  double get height
  Point<double> get center

As well as a bunch of extensions on Point which make things easier or do common mathematical things.

Point<double> rotate(T angle) =>
      Point(x * cos(angle) - y * sin(angle), x * sin(angle) + y * cos(angle));

Point<double> get normalized {
final m = magnitude;
return Point(x / m, y / m);
}

Point<double> pointTowards(Point<double> otherPoint,
    {double proportion = 0.5}) {
return this + (otherPoint - this) * proportion;
}

double dot(Point<double> other) => x * other.x + y * other.y;

Offset get offset => Offset(x, y);

Solandra Flutter

An early version is already available and quite usable: Solandra (Flutter). You can also see, thanks to Flutter's support for the web, a live demo. I did find it remarkable how a few demos I only ran on macOS could be very easily deployed to a website without any changes. The initial blank screen/load time is a little unfortunate; I haven't yet looked at how to improve it. It appears to cache, however, so subsequent loads are fast.

Dart/Flutter Tips

  • Dart is kind of Java-flavored JavaScript.
  • But it has some cool innovative features: a cascade operator .. allows you to chain many calls on one object, and every class is an interface (so no need for that common OOP thing of writing out both an interface and a class all the time).
  • You need semicolons at the end of lines.
  • The type goes in front of the variable or function.
  • But it often can be inferred, and you can use var for a variable, final for something you won't change, and const for something that could be created at compile time.
  • forEachIndexed exists, but you must import 'package:collection/collection.dart';
  • expanded is the Dart version of flatMap.
  • Dart/Flutter come with lots of built-in things, e.g., Point.
  • (2D) Point is generic, but I think for many creative code uses, it is easiest to use Point<double> everywhere.
  • Try to use standard things, like the existing Size class.
  • Return types should be given; void if it doesn't return something.
  • Custom drawing is not clipped by default; Solandra includes clipped() for this.
  • See also my article on Flutter in (late) 2021

Conclusions

Flutter seems remarkably well suited as a platform for creative coding. I can't think of an alternative that delivers on all of:

  • Wide cross-platform targets, with high quality.
  • Performance is good for fully custom, fully animated 2D drawing.
  • Hot reloading + Declarative UI.

If you are used to Processing or P5.js, you should give it a try for your next project. If you are completely new to creative coding it would likely be better to use something with more examples. While Dart is easier and better than Java, you will struggle to find examples of creative coding.