Hi r/scheme!

I've just released a moderate game (10,000+ lines of code) written entirely Scheme! I used the cross platform framework lambdanative which is based on the Scheme->C compiler gambit-c. The game is available on iOS/Android/Windows/Linux and OSX.

I've been developing the game in my free time (which is limited) for about three or four years. I thought this community might be interested in a post-partum about developing games in Scheme and using Lambdanative in particular.

First of all, I'd like to thank the team working on lambdanative - they were responsive and helpful. The framework worked for me! It helped turn this Schemer's dream of writing cross-platform code in a nice language into a reality.

That said, using lambdanative wasn't without rough patches. These were/are, roughly:

1. Library support

2. Cross platform issues

3. Debugging and macro-writing difficulties

4. No standard object system

I'll try to hit each point. Since an almost complete prototype for Corpse Wizard was written in Javascript (for the browser) before a rewrite and port to Scheme, I'll sometimes contrast the development experiences between the two systems.

What is Lambdanative?

A quick summary of what Lambdanative actually is provides useful context. Lambdanative is built on top of gambit-c which is a Scheme to C compiler. At its simplest, Gambit-c takes a Scheme source file and outputs a C file which then can be compiled into a shared object or executable with other pieces of C code.

Lambdanative, then, is a set of bindings which allow graphical and audio operations to be performed via a single Scheme api on all platforms. Lambdanative includes the substantial scripting required to automatically build the resulting executables for all platforms (though you need, for instance, to develop on an OSX machine to build for iOS or OSX). For the two portable platforms I targetted, this is accomplished by using the native code support for iOS and Android - the bindings are written in Objective C and C respectively, with the latter making calls to Java via the JNI.

Graphics for all systems are handled via some version of OpenGL, though most primitive operations are not exposed to the Scheme Programmer - you'd have to write a lot of binding code to use custom shaders, for instance.

Audio is more eclectic and therefor harder to work with. Where I did contribute patches to Lambdanative, it was to the end of resolving audio issues.

Library Support

With that summary out of the way, I can talk about the first issue I had working with Scheme and Lambdanative. As indicated above, Corpse Wizard was prototyped in Javascript before I converted and rewrote it in Scheme.

(You might ask at this point: why not use one of the many Javascript to portable/desktop frameworks out there instead of porting to Scheme? The answer is that Javascript performance is very dependent on the run time - my experience trying it out was that it was very hard to performance tune for all platforms simultaneously. Plus, JS engine versioning got complicated and none of the systems I tried made debugging easy. In contrast, Gambit-c provides a more understandable execution model (meaning optimizations which work on one platform tend to work as well on another), and I could debug and test on Desktop and trust my code would still run on portables (mostly)).

It can't be avoided: Javascript is an extremely active language with a great set of toolchains to help developer's quickly integrate a huge variety of library code. For instance, if you want to do A* pathfinding, you don't need to write that code yourself. There are plenty of off the shelf, serviceable libraries you can use.

In short, I had to write a lot of code in Scheme Corpse Wizard just to reach the point where I could start working - a lot of that code went into Lambdanative itself. I submitted patches for pathfinding, an entity-component-system framework and pattern matching (that last one wasn't in Javascript, obviously).

Like any discussion of the library problem in any language, we can ameliorate the issue by contributing code ourselves. But that takes time. It can be hard to beat popularity.

Cross Platform Support

While Lambdanative supports a nice set of platforms, I discovered that in many small ways, that support is not always complete. In every case the developers were extremely gracious, to say nothing of rapidly responsive, but these issues will get in your way from time to time.

Some examples: building on iOS (as of this writing) requires modifying, by hand, the Info.plist template for each game you create. This requires have a checkout of repository and fiddling with the contents.

Framerate also behaves different on each platform and requires changing code by hand to tune - code in multiple places in the repository, as it is in the native bindings, typically. Android frame rate doesn't even have a direct knob you can tweak.

Debugging and Macro Writing Difficulties

The single biggest difficulty I had with lambdanative is derives from Gambit-c and pertains to macros. It is unfortunately an easy trap to walk into, since its not advertised in any big way.

Gambit-c ships with and seems to favor a Common Lisp style defmacro form. However, if you write such macros then any error triggered by an error in the body (at any level) of such a macro, will silently kill your program without giving a line number or file. This makes debugging extremely difficult!

There are multiple solutions, however. The first is to just include and use syntax-case as a module. This is the solution I will prefer in the future, since I believe hygienic macros are just better and easier to use than Common Lisp style macros. However, there are issues with this approach!

The implementation of syntax-case that comes with gambit cannot coexist with gambit's own custom lambda expression syntax for optional, rest, and keyword arguments. By the time I diagnosed the issue, I had written about 10,000 lines of code, almost all of which used custom lambda expressions.

Luckily for me (and for people who prefer Common Lisp style macros -- you do you!) there is an alternative. Unfortunately, its undocumented, and a little hard to work out.

But it is possible to use gambit-c's own internals to write CL-style macros which preserve enough syntax information to get good error messages. See the note at the bottom of this article for an example.

About three quarters of the way through Corpse Wizard I had to stop and rewrite all my macro-based code, which included, at that time, an object system and a pattern matching macro.

Despite all that, from time to time I still encounter bugs which are hard to resolve, since they don't pitch good line number information. This seems to be inherent in gambit-c. There are instructions to build a compiler which produces more verbose, though presumably slower, builds, but how to integrate that procedure correctly with Lambdanative so far eludes me. The main idea here is expect to do a lot of printf style debugging.

No Standard Object System

The final challenge I encountered is that there isn't a standard object system in Scheme, nor even a standard way to represent one, as their is in Javascript. I tend to favor functional programming for many tasks, but in a game in particular, objects can be very useful units of organization. While most entities in the game follow the "composition over inheritance" maxim via an entity-component-system framework (which is a kind of object system), the artificial intelligences for each monster type are amenable to traditional single-inheritance object oriented style programming.

After a lot of different approaches with different object systems, I settled on SLIB's macroless object system. This choice was partially motivated by the problems with gambit-c's CL style macros.

My advice is thus: if you are going to use Lambdanative to do a project of any complexity, choose an object system first. Don't wait till you need one and then have to try a bunch of different choices.

My Contributions

As I worked on my game, I contributed a small amount of code back to Lambdanative. Some of that was patches to the back end to support multiplexed audio, but a few libraries might be of interest to anyone interested in developing their own game with Lambdanative.

1. scran is an entity-component-system framework based on the excellent Kran. This allows you to define your game objects in terms of components (like HP, target, etc) and systems, which constitute logic over sets of components. The idea is to make executing systems extremely efficient, so it can be done (in principle) on each frame. The trade off is that adding and removing components from an entity is somewhat slower, since system membership is calculated upon each addition or removal. In practice, this reflects typical usage in a game.

2. pathfinding This library implements a version of A* pathfinding on a grid.

3. Shadchen. This is a roughly ML-style pattern matcher that I carry around with me into whatever Lisp I am using. Its inspired by Racket's match form, but is much more naive. Usable, though!

Both scran and the pathfinding module emphasize speed and try to limit the amount of garbage generated, since we want to limit the GC pauses in a game. Shadchen isn't high performance, but is nice to use for high level code that doesn't need to be extremely fast.

The Game Itself

The game is called The Death of the Corpse Wizard. If you are interested in buying it, it can be found on itch.io, the App Store and the Google Play store. While the game is, at present, not open source, I'll open source it if and when I reach 500 sales across all the platforms.

The Death of the Corpse Wizard is sort of a combination of Robots and Shiren the Wanderer. The player uses a variety of movement and spatial abilities to try and survive wave after wave of monsters. There are about 10 different monster types with unique abilities and artificial intelligences. I've tried my best to make the game strategically engaging and fun.

Until then, I have an example game (Devil's Pong) which is open source which demonstrates how to use Lambdanative and Scran.

Conclusions

In conclusion, you can use Scheme to target multiple platforms for commercial games, as long as you are willing to put in a little extra elbow grease. I'd personally love to see Lambdanative get more attention, and hence more support, since I plan to continue to use it to develop games. Thanks for reading!

(PS - if you are interested in following news about Corpse Wizard or my other projects, follow my twitter, @jvtoups).

Appendix: Error Preserving CL-Style Macros in Gambit

Suppose you have a debugging macro called define*, which defines a function which prints its arguments to the debug log:

(define-macro (define* arglist #!rest body)
  (let ((function-name (car arglist))
        (args (gensym 'args-)))
    `(define (,function-name #!rest args)
       (define ,arglist ,@body)
       (display '(,function-name ,args))
       (newline)
       (apply ,function-name ,args))))

This is will unfortunately swallow any error messages, presumably by killing any trace information for the forms in body. We can fix this by rewriting our macro like this:

(##define-syntax define*
  (lambda (stx)
    (define (ensure-code o)
      (if (##source? o) (##source-code o) o))
    (let* ((syntax-body (cdr (ensure-code stx)))
       (first-form (##desourcify (car syntax-body))))
      (if (not (list? first-form))
      (##sourcify `(define ,@syntax-body) stx)
      (let* ((arg-list (ensure-code (car syntax-body)))
         (function-name (car (##desourcify (car syntax-body))))
         (arg-list-name (gensym 'arg-list))
         (body (cdr syntax-body)))
        (##sourcify 
          `(define (,function-name #!rest ,arg-list-name)
              (define ,arg-list ,@body)
              (display ',function-name ,arg-list-name)
                           (newline)
               (apply ,function-name ,arg-list-name))
            stx))))))

It isn't such a mystery what is going on here if you are a little familiar with syntax-case. We get a syntax object in and we have to manipulate the elements of that syntax object carefully, to preserve their meta-data. Finally, we return a new syntax object constructed via sourcify which uses the syntax meta-data of the original syntax object.

For the most part you can cargo cult this.

It’s our pleasure to announce the initial public release of

Structure and Interpretation of Computer Programs, JavaScript Adaptation
(https://sicp.comp.nus.edu.sg/)

A community effort led by Martin Henz and Tobias Wrigstad, this project provides the full content of the textbook classic Structure and Interpretation of Computer Programs by Harold Abelson and Gerald Jay Sussman with Julie Sussman, using sublanguages of JavaScript, instead of the language Scheme. We provide our SICP JS adaptation in three editions: a mobile-friendly interactive web edition, an interactive PDF edition and an e-book edition. Readers of the textbook can click on the programs and run them using the Source Academy [https://sourceacademy.nus.edu.sg/playground], a web-based programming environment that supports a collection of purpose-built language implementations of the JavaScript sublanguages Source §1, Source §2, Source §3 and Source §4 [https://sicp.comp.nus.edu.sg/source/], each of which are designed to serve the respective chapters of SICP JS.

SICP JS has been used by the National University of Singapore in the computer science freshman programming methodology course CS1101S [https://www.comp.nus.edu.sg/~cs1101s/] since 2012, and is maintained by the CS1101S community as an open source project. We share it with
educators and learners and welcome contributions and suggestions.

Full announcement and some fun examples: https://sicp.comp.nus.edu.sg/announcement.html

Martin Henz and Tobias Wrigstad

For fun this afternoon, I made a scheme program that prints a yin-yang to the terminal.

(define (fncircle px py x y r)
 (let ( (x (- x px))
        (y (- y py)) )
  (< (+ (* x x) (* y y)) (* r r))))

(define (xor a b)
 (and (or a b) (not (and a b))))

(define (fnyinyang x y)
 (cond
  ((or  (< x 0)  (< y 0)  (> x 1)  (> y 1))
   0)
  ((not (fncircle x y 0.5 0.5 0.5))
   0)
  (else
   (let* ( (which (> y 0.5))
           (y (- y 0.125 (if which 0.5 0))) )
    (if (fncircle x y 0.5 0.125 0.25)
     (- 2 (if (xor which (fncircle x y 0.5 0.125 0.0625)) 1 0))
     (- 2 (if (> x 0.5) 1 0)))))))

(define char-lookup (vector " " "." "#"))

(define (draw-yin-yang height)
 (let ( (width (* height 2)) )
  (do ( (y 0 (+ y 1)) ) ( (>= y height) )
   (do ( (x 0 (+ x 1)) ) ( (>= x width) )
    (let ( (xx (/ x width))
           (yy (/ y height)) )
     (display (vector-ref char-lookup (fnyinyang xx yy)))))
   (newline))))

(draw-yin-yang 50)

Today is Scheme's birthday, of a sort.  Today in 1975, SCHEME: An Interpreter for Extended Lambda Calculus was published as MIT AI Memo 349 by Gerald J. Sussman and Guy L. Steele, Jr.  Here's the abstract:

Inspired by ACTORS [Greif and Hewitt] [Smith and Hewitt], we have implemented an interpreter for a LISP-like language, SCHEME, based on the lambda calculus [Church], but extended for side effects, multiprocessing, and process synchronization. The purpose of this implementation is tutorial. We wish to: (1) alleviate the confusion caused by Micro-PLANNER, CONNIVER, etc. by clarifying the embedding of non-recursive control structures in a recursive host language like LISP. (2) explain how to use these control structures, independent of such issues as pattern matching and data base manipulation. (3) have a simple concrete experimental domain for certain issues of programming semantics and style.

Happy birthday, Scheme!

Thought some people here might be interested in a project I’ve been working on. I have a few applications written in guile scheme and wanted to help lower the barrier for newcomers to experiment/share/collaborate on scheme code. So far I’ve gotten something pretty usable and hope others find it interesting/useful!

Disclaimer: a lot of this app is written using the help of AI, mainly as I’m a dad with little free time but still want to advance some of these ideas. I definitely understand if the ai usage prevents you from feeling comfortable using the project. Thanks for your interest and feedback!

https://guileide.dev

The new beta of LIPS Scheme just got released. The evaluator was rewritten. It now supports continuations and TCO.

The implementation was inspired by JS-Scheme by Alex Yakovlev.

There is also a new quasiquote implementation based on Alen Bawden's paper "Quasiquotation in Lisp", a new macroexpand, and some performance improvements.

Full article about the release on the blog:

https://lips.js.org/blog/beta-continuations-and-tco

After releasing continuations, I've rediscovered the Chibi R7RS test suite.

I was working the whole week, and finally all tests are passing. I only commented out a few things.

One issue that stands out:

• delay-force - I'm not sure how it should work, so stream-filter doesn't work.

There are also some minor issues:

• Comments inside cons like (#;x . x) that should throw an error according to R7RS, but the dot operator in LIPS is a special getter that can appear as a first item in the list, so this is the same as (#;x |.| x).

• Missing D/L/S/F inside numbers like 1L2 (makes no sense in JS).

• Missing some error procedures like file-error? or read-error?

A Lisp Machine-style desktop that lives inside one host window and is written entirely in Chez Scheme. Every window, color, key binding, status-bar segment and running process is a live Scheme value. You can inspect it, redefine it and break it from a REPL that runs inside the world it changes.

If you're interested in helping with an effort to organize scheme.org, a new family of web sites for Scheme, please join us on the [email protected] mailing list. I'm re-posting the announcement here, but please subscribe here to participate in the discussion.

Welcome to the mailing list for the Scheme.org domain name.

First, some fundamentals. The registrant on file for Scheme.org is
Magnus Ahltorp from Sweden. The administrative and technical contact
is Arthur Gleckler (known to Schemers as the current SRFI editor). The
previous admin and tech contact was Shriram Krishnamurthi (known for
running Schemers.org among many other contributions). Shriram was
happy to hand over Scheme.org to people who have time to develop to
it, and would like to phase out Schemers.org in favor of Scheme.org
once the latter has been properly established. Magnus is also happy to
be hands-off but would like to retain ownership of the domain to
ensure that it stays non-commercial and does not track the behavior of
visitors. Magnus has kindly pre-paid the domain until 2029. The domain
registrar and DNS provider is Gandi.net.

On to the plans: The essay below is an attempt to lay out a particular
vision for Scheme.org. It has been hashed out by me and Arthur with
input from a handful of other Schemers. It advocates for an unusual
approach because Scheme is governed differently from most programming
languages. It is long and detailed; please bear with it, think about
it for a while, and tell us what you think on this list so we can get
a conversation started. Going forward, we want the administration of
Scheme.org to happen entirely in public so that everyone is clear on
where things stand with this important domain.

The title of this essay is chosen to make a particular point.

As programmers, we are used to fast-moving projects, changing plans
and informal customs. Getting software written at all is so hard that
we have little patience for elaborate social protocol.

We believe Scheme.org should be viewed as a fundamentally different
kind of project: a conservative one that prioritizes preservation and
planning even when that makes new development more difficult.

Scheme is not a new language. It comes with almost half a century of
history. It has a well established reputation, now guarded by us.

Scheme's continuity rests on a loosely organized network of volunteers
who come and go. While this is wonderful, the risk of key people going
out of reach is ever with us. We must take care not to jeopardize the
continued availability, maintenance and development of our resources.

Schemers are known to prize independent thought, sometimes to a fault.
Our most notable disagreement was the R6RS vs R7RS split. We should
take care to preserve neutral ground at the center of Scheme so that
adherents of different sub-communities can continue to work together
on shared interests even during times when there is widespread
disagreement over the future direction of Scheme. Likewise, we should
take care that no particular sub-community can alter the public image
of Scheme to the detriment of others.

Finally, we should protect the Scheme community from being co-opted by
commercial, academic or political interests for their own purposes.

All of the above concerns are fundamentally tied to the name itself:
Scheme. Whoever owns the name holds significant power over the public
image of the language and the ability of people to organize around it.
That is fine for languages headed by a definite person or group, but
Scheme has no owner. The closest thing we have is the Scheme Steering
Committee. They ratify the RnRS reports but are far too busy to tend
to the day-to-day matters of the language and its community. Running
the language community thus falls on its most avid users. That is us.

As the internet domain most obviously linked to Scheme's name,
Scheme.org is the one entity best placed to represent Scheme in the
world. With Scheme.org now in friendly hands, we should take this
opportunity to gather around it, make it the best we can, and do our
best to ensure it carries on to the next generation of Schemers.

Why think about the next 20 years? A lot can happen in a couple of
decades, and it's not guaranteed that any of us will still be involved
with Scheme after so much time. Setting our sights that far into the
future forces us to design a social and technical structure to outlast
ourselves. We can start with a template and gradually refine it. If we
do this right, the rate of change will slow down over time and the
community will increasingly trust in the stability of the structure.

Since humans are bad at predicting the future, the most likely success
formula is extreme simplicity in the core processes:

- We cannot have any rules that are hard to interpret since we may not
   be around in the future to give advice about them. Rules, procedures
   and responsibilities need to be simple and straightforward.

- We have to codify all knowledge in either code or written documents.
   We can't rely on word-of-mouth and knowing people who know things.

- We have to take maintainer fatigue into account. Scheme.org needs to
   have an administrator at all times, and it's a lot easier to get
   someone to commit to a job when it's easy and doesn't take much
   time. We can't have routine procedures that take a lot of effort.
   Likewise, training a new administrator has to be quick and easy.

- Scheme.org is predicated on the idea that Scheme, the DNS (Domain
   Name System) and the WWW will still be around 20 years from now.
   However, it's difficult to say what other tools will be popular. We
   should favor simple, well-known and platform-neutral tools without
   elaborate dependencies.

Moving further away from the core of Scheme.org, the stakes are lower
and we can be more relaxed. But as with the Scheme language itself, we
should put a great deal of thought and care into a core that will
stand the test of time.

On the technology front, we believe one trick in particular holds the
key to success: subdomains. The internet is naturally divided by DNS
delegation. This exactly mirrors a desirable governing structure for
Scheme.org: The root Scheme.org and its web front page should be
simple, conservative, and closely guarded -- like a constitution. Less
critical subdomains can afford to be more free-wheeling and
experimental about both their technical and social aspects.

Subdomains are simple to add and cost no extra money. They are a
convenient unit of division to work with in various networking,
security, and sysadmin contexts. We can freely make lots of them:

* r5rs.scheme.org
* r6rs.scheme.org
* r7rs.scheme.org
* srfi.scheme.org (should move late -- will be a big ordeal)
* docs.scheme.org
* pkgs.scheme.org
* lists.scheme.org
* files.scheme.org
* registry.scheme.org
* fantastic.scheme.org (your adventurous Scheme implementation here)
* ...

It's important to note that we could have as many as 50 subdomains and
the site would still be navigable. The front page can group them into
categories, with a short explanation next to each subdomain.

There would naturally be two kinds of subdomains:

* Those owned by a particular group based around a given standard,
   implementation, or tool. Each subdomain of this kind should be
   controlled by the group in question.

* Those that are community-owned resources about a particular topic,
   similar to the new Scheme Topics mailing lists at srfi.schemers.org
   and our many GitHub organizations. They should operate by consensus,
   with decision making on a public forum such as a mailing list, and
   with the top level Scheme.org administrators acting as last-resort
   tie breakers for any disputes.

Each subdomain should be responsible for setting up and maintaining
its own servers. The top level Scheme.org administration should only
be tasked with updating DNS records as requested, and with maintenance
of the Scheme.org front page. Delegating all other responsibilities to
subdomain admins will naturally ensure the fault tolerance of
Scheme.org as a whole and enable a light workload for the people
committed to be the top level administrators.

One important angle missing from the above treatment is the legal one.
No amount of goodwill or technical savvy can help us if we are no
longer the registered owner of Scheme.org. Scheme is a common English
word, and Scheme.org could be sold for upwards of 10k USD if someone
was either greedy or desperate for money. It could also be taken over
by an organization incompatible with our goals.

To that end, a few of us have pondered whether it might be a good idea
to set up a formal non-profit association to act as the legal owner of
Scheme.org and any other important internet properties related to
Scheme that people wish to donate to it. We may not have anyone in the
Scheme community who is excited about the paperwork involved, but the
stakes are high enough that if the cost is not prohibitive this step
is worth taking anyway.

Ideally Scheme.org would become so reliable that Scheme implementors
could feel as comfortable hosting their implementation's primary
website at fantastic.scheme.org as they would at their own separate
domain fantastic-scheme.org. There would be nothing like gathering
implementations under the umbrella of Scheme.org to signal the
cohesion and organization of the Scheme community to outsiders -- a
level of cohesion that is probably unmatched by any other significant
language. But authors hold their implementations dear, and major ones
will not take such a drastic step without a deep trust in the social
and legal basis of Scheme.org. If we work hard at building that trust
for a number of years, this may eventually be our ultimate prize. But
even without such an accomplishment, every extra resource we add under
Scheme.org builds the reputation of Scheme as a collaborative
community and provides direct benefit to users who can thus more
easily find their way around Scheme.

If you're interested in helping us bring Scheme.org and its subdomains
to life, please join this discussion. We're looking for volunteers to
create and maintain Scheme.org and interesting and useful subdomains,
and to help us figure out the process for organizing this effort for
the long haul.