From: Tim Bradshaw <tfb@cley.com>
Newsgroups: comp.lang.lisp
Subject: Re: Question: macros and lambdas
Date: 27 Sep 2002 12:56:17 +0100
Organization: Cley Ltd <www.cley.com>
Lines: 96
Sender: tfb@lostwithiel
Message-ID: <ey3ptuz1w9q.fsf@cley.com>
References: <56Vk9.2190$sh4.101485@newsfep2-win.server.ntli.net>
NNTP-Posting-Host: lostwithiel.cley.com
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Trace: news.demon.co.uk 1033127793 13400 212.240.242.98 (27 Sep 2002 11:56:33 GMT)
X-Complaints-To: abuse@demon.net
NNTP-Posting-Date: Fri, 27 Sep 2002 11:56:33 +0000 (UTC)
User-Agent: Gnus/5.0808 (Gnus v5.8.8) XEmacs/21.1 (Cuyahoga Valley)
Path: news.dca.net!yellow.newsread.com!netaxs.com!newsread.com!grr!newshub.northeast.verio.net!verio!kibo.news.demon.net!news.demon.co.uk!demon!lostwithiel!nobody
Xref: news.dca.net comp.lang.lisp:38747

* Jacek Podkanski wrote:
> I'm quite new to Lisp. At the moment I'm trying to understand some of its 
> concepts. Somewhere I seen information that everything that can be done 
> with macros can be done with lambdas. Is this correct? Can anybody point 
> out differences between the two?

I think that the term `everything can be done with x' needs to be
clarified.  As it stands it is altogether too close to some kind of
Turing equivalence notion which is not really useful in context.

Essentially macros in Lisp are source to source transformations which
are themselves specified in Lisp.  These transformations are rather
easy to express because of the facts that Lisp source code is
represented as a Lisp data structure; Lisp has so little syntax - so
there is no upfront commitment to what a given bit of source `means'
in the sense that `{...}' is a block in C; and finally that lisp does
not have a distinction between expressions and statements in the
language.  Thus, rather than the limited `string-replacement' macros
that, say, C has, Lisp macros allow almost completely general rewrites
of the structure of a bit of source code, and the code that implements
the macro works in terms of the structure of the source, not some
stringy representation.

All of this makes Lisp macros extremely powerful and flexible tools.
The result of having them is that you can extend the `syntax' of the
language to introduce, say, new control constructs or binding forms,
or ultimately to create a whole lisp-like language on top of Lisp.
And all of this can be done incrementally - there's never a point
where you have to sit down with yacc and lex and write a parser for
your new language.

The result of this is that substantial Lisp programs tend to evolve
all sorts of domain-specific `slang' which expresses very succinctly
the concepts used in the domain.  For instance, in a system I've
written recently I have things like:

    (with-session ()
      ;; in here there is a notion of `session variables' which are a
      ;; bit like shell variables
      ... code ...)

    (with-io-session ()
      ;; in here you can open and access named `output viewers' which correspond
      ;; to windows on the screen (typically) which display random
      ;; text you send them.  At the end of this scope all these
      ;; windows will be closed (unless the IO session has been `captured').
      ... code ...)

    ;;; define a hook named FOO
    (define-hook foo)

    (define-hook-named bar foo ()
      ;; define a bit of code on the FOO hook called BAR.  If this is
      ;; evaluated more than once then the BAR code will be replaced
      ;; with the new stuff.  Everything gets compiled if the source
      ;; file is compiled.
      ... code ...)

    ;;; Run all the code on the FOO hook, passing no arguments.
    ;;; This isn't a macro, it's just a function...
    (run-hook 'foo :args '())

Macros are sometimes fairly fiddly to write as you have to both think
in a fairly unconventional way - in terms of things that take source
and rewrite it &c, and you also have to be aware of various issues
which the system might not protect you from, such as name capture
issues (some macro systems do protect you from these issues, and you
will see incessant bickering about how ones that don't are not good
&c).  Writing macros is programming language design, and that is not
easy.

OK, so now the equivalence stuff.  Well for one, clearly macros don't
buy you some new formal power - the language is just turing equivalent
anyway.  But another thing is that, if you had a Lisp without a macro
system, you could pretty much write one.  For instance, if you were
happy with a file-based system, you could write a program which read
(using READ) a source file, form by form, and then processed it with a
function called, say, EXPAND which implemented the user-written macro
system, and then evaluated the result.  The same could clearly be done
for an interactive sytem:

    (loop
      (print (simple-eval (expand (read)))))

where SIMPLE-EVAL is something that doesn't understand macros, and
EXPAND is the user macro expander function.  EXPAND is fiddly to
write, but not that fiddly.

So if you didn't have macros, you could implement them - the really
crucial thing is READ, and source code being data.  Indeed this is
almost certainly how the first Lisp macro systems *were* written.

But just because you *could* do it doesn't mean you should *want* to,
of course.

--tim