/paper/yoko-old.tex

\documentclass{sigplanconf}

\input{preamble}

\begin{document}

\maketitle

\section{Introduction}

\todo{page limit = 12}

\todo{distinguish occurrences of ``\ig '' between the approach, the generic
  view, and the library}

\todo{Cite \url{http://dreixel.net/research/pdf/fcadgp.pdf}}



Generic programming libraries deliver, in the language, a degree of reuse that
is conventionally attainable only by metalingual capabilities such as special
support from compilers and generative programming. The \ig\ Haskell library,
for example, generically defines a function testing for equality, a function
for printing, and an ``empty'' value, all of which can be instantiated for most
data types with minimal integration effort \citep{instant-generics}. Moreover,
the library user can define their own extensible generic values. The common
trait of these functions is that they can be defined in terms of the type
structure of their domains and ranges.

This paper presents the \yoko\ library, \todo{\ig\ is strictly defined in a way
  which forbids extention: it's an approach framework that we're working
  in. We're extending its demonstration on Hackage.} which extends the
\ig\ approach both (1) to enable more exact types for generically defined
values and also (2) to generically define functions that require more
structural information than existing generic programming libraries provide.
The foundation of \yoko\ is a generic view \citep{generic-view} called
\emph{sum-of-constructors}, which enhances the common \emph{sum-of-products}
view. While sophisticated use of sum-of-products can almost emulate
sum-of-constructors, sum-of-products falls short in a way that violates
encapsulation of the original data type's representation, thereby sacrificing
both modularity and lucidity. Beyond these essential benefits to software
quality, the enhanced view accomodates a more complete reflection of data type
declarations. For example, \yoko\ thereby enables a generic treatment of
polymorphic variants \citep{poly-variants} and transformation between similar
data types.

\begin{table}[h]
\begin{tabular}{l|cc}
\textbf{Syntactic Sort} & \textbf{\# of Types} & \textbf{\# of Constructors} \\
\hline
Type & 4 & 15\\ % (^Type^, ^TyVarBndr^, ^Kind^, ^Pred^), nominal: (^ForallT^, ^VarT^, ^ConT^, ^ClassP^)
Declaration & 12 & 37\\ % (^Dec^, ^Con^, ^Strict^, ^Foreign, Callconv^, ^Safety^, ^Pragma^, ^InlineSpec^, ^FunDep^, ^FamFlavour^, ^Fixity, FixityDirection^)
Expression & 8 & 41\\ % (^Exp^, ^Match^, ^Clause^, ^Body^, ^Guard^, ^Stmt^, ^Range^, ^Lit^)
Pattern & 1 & 14\\ % ^Pat^
\textbf{Total} & 25 & 107
\end{tabular}
\caption{The Template Haskell version 2.6 AST.\label{tab:th-ast}}
\end{table}

The statistics listed in \tref{Table}{tab:th-ast} for the Template Haskell
version 2.6 AST characterize the burden of each new data type. Each data type
in a Haskell compiler's pipeline would be approximately as large as the
Template Haskell AST. \todo{How many such data types in GHC?} Specifying
functions over such large ASTs is a daunting task that can be significantly
mollified by leveraging generic programming as much as possible.

Existing generic programming libraries such as \ig\ successfully mitigate the
first burden by nearly direct reuse of the generically defined functions for
each of the many types. The second burden, however, has so far remained beyond
the purview of generic programming. The tedious cases of transformations
require constructor-agnostic bookkeeping, structural recursion, and a
correspondence between the similar constructors of adjacent types in the
pipeline. Existing techniques can muster the first two of these, but are
incapable of automatically identifying corresponding constructors except under
prohibitively degenerate circumstances.

By supporting generic transformations between similar types, \yoko\ reduces the
cost of encoding invariants as data types. The key to its support is its
ability to identify corresponding constructors in distinct data types, which is
a reflective capability derived from the sum-of-constructors view. By reducing
the cost of encoding invariants in the type system, the \yoko\ generic
programming library enables higher assurance of compiler correctness.

This paper conveys the following contributions.

\begin{enumerate}
\item Demonstrations of basic insufficiencies of the sum-of-products view as
  used in the \ig\ Haskell package.
\item The incremental introduction of \yoko's enhancements redressing those
  insufficiencies.
\item A definition of a realistic transformation using \yoko's unique
  capabilities. Lambda lifting transforms from an object language with
  anonymous functions to an object language without them. In our definition
  this transformation, all syntactic constructs that do not directly involve
  binding are handled generically.
\item The lambda lifting transformation's generic treatment of its non-nominal
  syntactic constructors can be factored out. The resulting transformation has
  a purely generic definition using only the \yoko\ API, and thus it can be
  directly reused in the definition of other transformations.
\end{enumerate}

\section{Objective}
\label{sec:objective}

Lambda lifting \citep{lambda-lifting} is a transformation in many compiler
pipelines that maps from an object language with anonymous functions to an
object language without them. The declarations listed in
\tref{Figure}{fig:lambda-lift-sig} include the two data types modeling the
terms of each object language as well as the ^lambdaLift^ function. This
section explains the invariants encoded by the two data types and motivates the
generic treatment of most constructors in the definition of lambda lifting.

\begin{figure}[h]
\begin{haskell}
lambdaLift :: AnonTerm -> TopProg

module Common where
  -- object language types, including at least arrows
  data Type = ...

module AnonymousFunctions where
  data AnonTerm = Lam Type AnonTerm | Var Int
                | Let [Decl] AnonTerm
                | ... -- non-nominal constructors
  newtype Decl = Decl Type AnonTerm

module TopLevelFunctions where
  data TopTerm = DVar Int | Var Int
               | ... -- same non-nominal constructors
  type FunDec = ([Type], Type, TopTerm)
  data TopProg = Prog [FunDec] TopTerm
\end{haskell}
\caption{The signature of lambda lifting.\label{fig:lambda-lift-sig}}
\end{figure}

\subsection{The Signature of Lambda Lifting}

The ^AnonTerm^ type models the terms of higher-order functional language with
anonymous functions. It includes the ^Lam^ and ^Var^ constructors for de
Bruijn-indexed lambdas and variable occurrences as well as the ^Let^
constructor and ^Decl^ type that models a non-recursive let form with multiple
nested declarations. The ^Let^ constructor incurs mutually recursion with the
^Decl^ type. For our purposes, the type is also assumed to include any number
of other constructors modeling \emph{non-nominal} syntactic constructs, those
that do not involve binding or variable occurrences. Function application,
tuples, lists, and literal values of base types, for example, are
non-nominal. The ^TopProg^ type, on the other hand, models the terms of a
higher-order functional program in which functions can only be named and
globally declared. Such a program is modeled as a pairing of a
topologically-ordered list of top-level function declarations and a body term;
the declaration bodies and the main body are terms without anonymous functions
as modeled by the ^TopTerm^ type. As the successor of ^AnonTerm^ in the
pipeline of data types, ^TopTerm^ retains most of ^AnonTerm^'s constructors. In
order to encode its invariant, however, it drops the ^Lam^ and ^Let^
constructors. For simplicity, we assume the corresponding constructors of both
^AnonTerm^ and ^TopTerm^ have the same name (hence the separate modules) and
the same fields after accounting for the substitution of ^TopTerm^ for
^AnonTerm^.

The ^AnonTerm^ and ^TopProg^ types encode the invariants that the modeled terms
adhere to the respective grammars with and without lambdas. For a Haskell AST,
these are not particularly impressive invariants. They are, however, quite
pragmatic---the presence or absence of anonymous functions is a significant
property to statically guarantee---and require a transformation inexpressible
with existing generic programming techniques.

Among common language features, only nominal ones are essential to lambda
lifting. The nub of the ^lambdaLift^ function's semantics is accordingly
present only in its cases for the ^Lam^, ^Var^, and ^Let^ constructors. As
described in the introduction's second burden of a pipeline with numerous data
types, the lambda lifting of non-nominal constructors merely maintains some
bookkeeping and recurs through subterms, mapping each ^AnonTerm^ constructor to
the obvious counterpart in ^TopTerm^. In this case, the bookkeeping involves
generating the list of top-level declarations corresponding to the lifted
lambdas. This is naturally implemented in
\tref{Section}{sec:lambda-lift-definition} with a writer monad. It is the
automated identification of corresponding constructors which \yoko\ uniquely
enables.

\subsection{Discussion}

The compound first field of ^Let^ and the corresponding mutual recursion makes
^AnonTerm^ a realistic example.

%% Modern generic programming approaches continue to depend on metalingual
%% capabilities (\eg\ Template Haskell), but only for convenience: the libraries
%% derive their genericity from declarations within the langauge. \todo{TODO} TODO
%% what's the benefit of that? Better integration with other language features?
%% Portability?

\section{\ig\ Background}

\todo{grammar for the representation types}

The \ig\ approach \citep{instant-generics} underlies \yoko's generic
programming capabilities. The approach derives its genericity from two major
Haskell language features: type classes and type families
\citep{type-families}. We demonstrate a slight simplification of the library
with an example type and two generically defined functions in order to set the
stage for \yoko's enhancements. We retrofit our own vocabulary to the concepts
underlying the \ig\ Haskell declarations.

\tref{Figure}{fig:instant-generics} lists the core \ig\ declarations. In this
approach to generic programming, any value with a \emph{generic semantics} is
defined as a method of a type class, called a \emph{generic class}. That
method's generic semantics is declared as instances of the generic class for
each of a small finite set of \emph{representation types}: ^Var^, ^Rec^,
\etc. The ^Rep^ type family maps a data type to its sum-of-products structure
\citep{sum-of-products} as encoded with those representation types. A
corresponding instance of the ^Representable^ class converts between a type and
its ^Rep^ structure. Via this conversion, an instance of a generic class for a
data type can delegate to the generic definition by invoking the method on the
type's structure. Instances of a generic class, however, are not required to
rely on the generic semantics: they can use it partially or completely ignore
it.

\subsection{The Sum-of-Products Generic View}

Each representation type models a particular structure in the declaration of a
data type. The ^Rec^ and ^Var^ types represent occurrences of types in the same
mutually recursive family as the represented type (roughly, its binding group)
and any non-recursive occurrence of other types, respectively. Sums of
constructors are represented by nestings of the higher-order type ^:+:^, and
products of fields are represented similarly by ^:*:^. ^U^ serves as the empty
product. Since an empty sum would represent a data type with no constructors,
its usefulness is questionable. The representation of each constructor is
annotated by means of ^C^ to carry a bit more reflective information in ^C^'s
phantom type parameter. The ^:+:^, ^:*:^, and ^C^ types are all higher-order
representations in that they expect representations as arguments. If Haskell
supported subkinding \citep{promotion}, these parameters would be of a subkind
of ^*^ specific to representation types. Since parameters of ^Var^ and ^Rec^
are not supposed to be representation types; they would have the standard ^*^
kind.

\begin{figure}[h]
\begin{haskell}
type family Rep a

data Var a = Var a
data Rec a = Rec a
data U = U
data a :*: b = a :*: b       
data C c a = C a 
data a :+: b = L a | R b

class Representable a where
  to :: Rep a -> a
  from :: a -> Rep a

class Constructor c where
  conName :: C c a -> String
\end{haskell}
\caption{Core declarations of the \ig\ library.}
\label{fig:instant-generics}
\end{figure}

Consider a simple abstract syntax for expressions denoting arithmetic sums,
declared as ^Exp^.

\begin{haskell}
data Exp = Const Int | Plus Exp Exp
\end{haskell}

\noindent An instance of the ^Rep^ type family maps ^Exp^ to its structure as
encoded in terms of the representation types.

\begin{haskell}
type instance Rep Exp =
  C Const (Var Int) :+: C Plus (Rec Exp :*: Rec Exp)

data Const; data Plus
instance Constructor Const where conName _ = "Const"
instance Constructor Plus where conName _ = "Plus"
\end{haskell}

The ^Const^ and ^Plus^ types are considered auxiliary by \ig, added as an
afterthought to the sum-of-products view in order to define another class of
values generically---^Show^ and ^Read^ in particular. Since \yoko\ uses them in
the same reflective capacity, but to a much greater degree, we designate them
\emph{constructor types}. Each constructor type corresponds directly to a
constructor from the represented data type (much like promotion
\citep{promotion}).

The ^Const^ constructor's field is represented with ^Var^, since ^Int^ is not a
recursive occurrence. The two ^Exp^ occurrences in ^Plus^ are recursive, and so
are represented with ^Rec^. The entire ^Exp^ type is represented as the sum of
its constructors' representations---the products of one and two fields,
respectively---with some further reflective information provided by the ^C^
annotation. The ^Representable^ instance for ^Exp^ follows directly from the
involved signatures.

\begin{haskell}
instance Representable Exp where
  from (Const n) = L (C (Var n))
  from (Plus e1 e2) = R (C (Rec e1 :*: Rec e2))
  to (L (C (Var n))) = Const n
  to (R (C (Rec e1 :*: Rec e2))) = Plus e1 e2
\end{haskell}

\subsection{Two Generic Definitions}

We generically define an equality test and the generation of a minimal
value. For equality, we reuse the ^Eq^ class as the generic class.

\begin{haskell}
instance Eq a => Eq (Var a) where
  Var x == Var y = x == y
instance Eq a => Eq (Rec a) where
  Rec x == Rec y = x == y
instance (Eq a, Eq b) => Eq (a :*: b) where
  x1 :*: x2 == y1 :*: y2 = x1 == x2 && y1 == y2
instance Eq U where _ == _ = True
instance (Eq a, Eq b) => Eq (a :*: b) where
  L x == L y = x == y
  R x == R y = x == y
  _ == _ = False
instance Eq a => Eq (C c a) where
  C x == C y = x == y
\end{haskell}

\noindent With these instance declarations, ^Eq Exp^ is immediate. As
\citeauthor{fast-and-easy} show, the GHC inliner can be compelled to optimize
away much of the representational overhead.

\begin{haskell}
instance Eq Exp where x == y = from x == from y
\end{haskell}

The method of the ^Empty^ generic class generates a minimal value\footnote{Note
  that \inlineHaskell{empty} is not a function. This is why we avoid the term
  ``generic function''.}.

\begin{haskell}
class Empty a where empty :: a

instance Empty Int where empty = 0
instance Empty Char where empty = '\NUL'
...
\end{haskell}

\noindent In the generic definition of ^empty^, it may seem odd to define an
instance for ^Rec^, since recursion seems contrary to minimality. This instance
is ultimately necessary for two reasons. First, in a mutually recursive family
of data types, one sibling might not have any non-recursive fields; to generate
its minimal value requires recursion to reach any sibling that has a
constructor capable of non-recursion. Second, the ^Rec^ instance enables a
reasonable use of ^Empty^ for coinductive data types, in which corecursion is
inevitable.

\begin{haskell}
instance Empty a => Empty (Var a) where
  empty = Var empty
instance Empty a => Empty (Rec a) where
  empty = Rec empty
instance (Empty a, Empty b) => Empty (a :*: b) where
  empty = empty :*: empty
instance Empty U where empty = U
instance Empty a => Empty (C c a) where
  empty = C empty
\end{haskell}

The ^Empty^ instance for ^:+:^ must prefer a summand (i.e.  a constructor)
capable of non-recursion. In order to do so, the auxiliary class ^HasRec^
provides a means for checking if a representation value involves recursion. The
instances for ^Var^ and ^Rec^ answer the predicate directly; the other
representation types' instances structurally recur.

\begin{haskell}
class HasRec a where hasRec :: a -> Bool

instance HasRec (Var a) where hasRec _ = False
instance HasRec (Rec a) where hasRec _ = True
instance (HasRec a, HasRec b) => HasRec (a :*: b) where
  hasRec (a :*: b) = hasRec a || hasRec b
instance HasRec U where hasRec _ = False
instance (HasRec a, HasRec b) => HasRec (a :+: b) where
  hasRec (L x) = hasRec x
  hasRec (R x) = hasRec x
instance HasRec a => HasRec (C c a) where
  hasRec (C x) = hasRec x
\end{haskell}

\noindent The ^Empty^ instance for ^:+:^ uses ^hasRec^ to avoid recursion when
possible. Laziness and the non-strictness of ^hasRec^ for ^Var^, ^Rec^, and ^U^
prevents the use of ^empty^ in the condition from diverging. For coinductive
data types, ^empty^ will generate instead an infinite nesting of the last
constructor, or the cyclic analog for a mutually corecursive family.

\begin{haskell}
instance (HasRec a, Empty a, Empty b
         ) => Empty (a :+: b) where
  empty = if hasRec lempty then R empty else L lempty
    where lempty = empty :: a
\end{haskell}

The ^Empty^ instance for ^Exp^ is a straight-forward delegation to the generic
definition and always yields ^Const 0^.

\begin{haskell}
instance Empty Exp where empty = to empty
\end{haskell}

\begin{note}{Nick}{Make \inlineHaskell{HasRec} static}

Let's not stoop to a dynamic predicate when we don't need to.

\begin{enumerate}
\item Search instead for the first constructor with no ^R^s in its structure.
\item Beware mutual recursion: some types need to traverse a finite number of
  ^R^s in order to reach a sibling type that has a constructor with no
  recursive fields.
\item There's a couple ways to avoid cycles in that search: keep a list of
  visited types (or better yet, typenames, irrespective of type parameters), or
  just limit the depth of the search to the size of the sibling set (assuming
  it's not infinite, as for nested types)
\item Flourish: choose the path to a non-recursive constructor with the
  smallest number of total fields?
\item Lastly, fallback gracefully for necessarily infinite types.
\end{enumerate}
\end{note}

As demonstrated with ^==^ and ^empty^, generic definitions---\ie\ the
corresponding instances for the representation types---provide an easily
invocable default behavior. If that behavior suffices for a representable type,
then instantiation of the method on the type's representation provides a simple
way to define the methods in an instance of a generic class for the
representable type. If a particular type needs a distinct ad-hoc definition of
the method, then that type's instance can use its own specific method
definitions, relying on the default generic definition to a lesser degree or
even not at all.

\section{\yoko's Enhancements}
\label{sec:enhancements}

The \yoko\ library extends \ig\ with three principal enhancements.

\subsection{Representation of Compound Fields}
\label{sec:compound-fields}

For the represention of data types with compound fields (\eg\ containing lists
and tuples), the paucity of the \ig\ representation types precludes precise use
of the ^Rec^ representation type. \todo{\ig\ isn't a view, sum-of-products is.}
Though our semantics for ^Rec^ is determined by recursive occurrences of types,
its semantics in \ig\ allows it to represent any type that contains recursive
occurrences. Other wise, it could not represent types with compound
fields. Consider representing a type such as ^Exp2^, where ^Const2^ optionally
takes another expression as a second argument.

\todo{Can we demonstrate mutual recursion? Perhaps Odd/Even lists?}

\begin{haskell}
data Exp2 = Const2 Int (Maybe Exp2) | Plus2 Exp2 Exp2

data Const2   ;   data Plus2

type instance Rep Exp2 =
  C Const2 (Var Int :*: Rec (Maybe Exp2)) :+:
  C Plus2 (Rec Exp2 :*: Rec Exp2)
\end{haskell}

\noindent Note that the argument to ^Rec^ in the representation of the ^Const2^
constructor must be the entire field, since the representation types include no
means of navigating the components of the field in order to apply ^Rec^ only to
the occurrence of ^Exp2^. The danger of simply enlisting ^Maybe^ as an ad-hoc
representation type to represent the field directly as ^Maybe___(Rec___Exp2)^
is discussed at the end of this subsection.

This imprecise use of ^Rec^ remains predominantly efficacious because the
declaration of instances does provide a means of navigating the composition of
the field. For example, the generic definition of ^Eq^ suffices for ^Exp2^,
since ^Maybe^ has a standard ^Eq^ instance defining equality in terms of its
type parameter (^Eq a^). This would not work if the ^Eq^ instance for ^Rec^
actually depended on the particular semantics of ^Rec^ as representing a
recursive occurrence, as is the case with the ^HasRec^ generic class.

For both ^==^ and ^empty^, the ^Rec^ and ^Var^ instances are exactly the
same. For ^hasRec^, however, the two instances differ in accord with the
essence of the semantics of ^hasRec^. It is therefore sensitive to imprecise
use of ^Rec^; in particular, the result of the call to ^hasRec^ in the ^empty^
method for the ^:+:^ type indicates that the second field of ^Const2^
recurs. Unfortunately, this results in ^empty___::___Exp2^ generating an
infinite nesting of ^Plus2^ instead of the obvious
^(Const2___0___Nothing)^. Because the representation of that field uses ^Rec^,
the corresponding ^hasRec^ method returns ^True^ without regard for the ^Maybe^
type containing the actual recursive occurrence in that field. In this way, the
imprecise use of the ^Rec^ representation type has compromised the definition
of ^empty^ for ^Exp2^.

A more precise alternative reserves the use of ^Rec^ for the actual recursive
occurrences themselves. In order to do so, \yoko\ enlarges the universe of
types that are representable with precise use of ^Rec^ by extending the
representation types with two types for representing applications of
^*___->___*^ and ^*___->___*___->___*^ types (\eg\ lists and tuples).

\begin{haskell}
newtype Par1 f c = Par1 (f c)
newtype Par2 ff c d = Par2 (ff c d)
\end{haskell}

\noindent The resulting universe of representable types is still incomplete,
but we suppose it includes significantly more of the data types commonly
declared in Haskell programs, including ^Exp2^. With the new representation
type, the second field of the ^Const2^ constructor now uses the ^Rec^ type only
for the recursive occurrence: ^Par1___(Maybe___(Rec___Exp2))^.

Adding two representation types is a simple enhancement; its only cost is the
corresponding additional instances for each generic class. Indeed, parametric
types such as lists and tuples could themselves be considered representation
types on an ad-hoc basis, but this can lead to ambiguity of its own. The
\ig\ library does not explicitly preclude such ad-hoc representation types, but
its omission of instances of ^HasRec^ for lists and ^Maybe^ seemingly endorses
the use of ^Rec^ to represent compound fields. Precise use of the ^Rec^
representation type enables accordingly more precise use of generic definitions
in \yoko\'s more advanced type programming.

General ^Par1^ instances tends to rely on a corresponding class parameterized
over ^*___->___*^ types, such as the ^Functor^ class. Indeed, the ^Par1^
instance for ^HasRec^ will use the ^Foldable^ class. General instances for the
^Par2^ type similarly rely on a corresponding class parameterized for
^*___->___*___->___*^ types. However, in many cases, including ^Eq^ and
^Empty^, the generic class for parameterized over ^*^ types can be re-used
directly. Otherwise, if no general higher-kinded class exists and the generic
class itself cannot be reused, an ad-hoc higher-kinded class must be declared.

\begin{haskell}
instance Eq (f a) => Eq (Par1 f a) where
  Par1 x == Par1 y = x == y
instance Empty (f a) => Empty (Par f a) where
  empty = Par1 empty
instance (Foldable f, HasRec a) =>
  HasRec (Par1 f a) where
  hasRec (Par1 x) = Data.Foldable.any hasRec x

instance Empty (Maybe a) where empty = Nothing
\end{haskell}

\noindent Now constraints over the representation of ^Exp2^ ultimately incur
corresponding constraints over ^Maybe^. As such, the more precise ^hasRec^
yields ^(Const2___0___Nothing)^ for ^Exp2^, as expected.

While it may seem attractive to reuse the ^HasRec^ class for the ^Par1^
instance as with ^Eq^ and ^Empty^, such a ^HasRec___(Par1___f___a)^ instance
with the context ^HasRec___(f___a)^ would be unsound. In a vacuum, it is
obvious that the correct definition of ^hasRec^ for ^Maybe^ is ^const___False^;
the ^Maybe^ type is not recursive. This, in turn, spoils the hypothetical
^Par1^ instance. For example,
^hasRec___(Const2___5___(Just___(Const2___...)))^ would incorrectly reduce to
^False^.

The key insight is that the semantics of ^hasRec^ is entirely dependent on its
type parameter. The unsound ^HasRec^ instance for ^Par1^ changes the head of
the type parameter to ^f^. If ^f^ were a representation type, this would be
fine, because representation types serve as a proxy for the represented nominal
type. However, according to the semantics of ^Par1^, ^f^ is not a
representation type: it is an unrestrained ^*___->___*^ type. Without using
^Par1^ to delimit the representation, there would be no opportunity to enable
the transition from ^HasRec^ to ^Foldable^. This is why adopting ^Maybe^ as an
ad-hoc representation type is dangerous.

\subsection{Delayed Representation of Constructors}
\label{sec:granularity}

The generic view of data types in \yoko\ delays the representation of
constructors as products of their fields. We motivate this as an enhancement by
exploring a hypothetical application of generic programming to mitigate a large
number of constructors. Generic programming is most obviously useful when
dealing with large data types, rapid prototyping of data types, or both. With
large data types, it is also likely that a minority of a type's constructors
deserve interesting deviation from the generic definition. Because \ig\ only
represents the entire data type, it struggles to delegate to the generic
definition on a per constructor basis.

Consider enhancing the ^Exp^ type's ^Const^ constructor to support various
encodings of numerical constants. The essence of this example is that the
semantics of that constructor no longer corresonds so directly to its
structure.

\begin{haskell}
data Encoding = Base Int | Roman | ...
decode :: Encoding -> String -> Int

data Exp3 = Const3 Encoding String | Plus3 Exp3 Exp3
\end{haskell}

We declare the ^Constants^ generic class with the ^constants^ method for
collecting the values of all numerical constants occuring in a value of a data
type. Again, this use of generic programming is hardly justified for ^Exp3^
itself, but imagine it is a larger data type with numerous
constructors---perhaps one for each arithmetic operation: subtraction,
multiplication, \etc. The generic definition of ^Constants^ merely recurs,
collecting constants from the fields.

\begin{haskell}
class Constants a where constants :: a -> [Int]

instance Constants a => Constants (Var a) where
  constants (Var x) = constants x
instance Constants a => Constants (Rec a) where
  constants (Rec x) = constants x
instance (Constants a, Constants b) =>
  Constants (a :+: b) where
  constants (L x) = constants x
  constants (R x) = constants x
instance (Constants a, Constants b) =>
  Constants (a :*: b) where
  constants (x :*: y) = constants x ++ constants y
instance Constants U where constants _ = []
instance Constants a => Constants (C c a) where
  constants (C a) = constants a
\end{haskell}

The semantics of the ^Const3^ constructor and the semantics of the ^Constants^
class overlap in such a way that the generic definition is incorrect for that
constructor: the intent is not to count the numerical constants occurring in
the ^Encoding^ and ^String^ fields. Therefore, the ^Constants^ instance for
^Exp3^ must treat ^Const3^ with ad-hoc behavior and only delegate to the
generic definition for the other constructors.

\begin{haskell}
-- NB speculative: ill-typed
instance Constants Exp3 where
  constants (Const3 enc s) = [decode enc s]
  constants e = constants (from e)
\end{haskell}

The ill-typedness of this obvious instance is due both to the coarseness of the
sum-of-products interpretation and a limitation of the Haskell type system. The
resulting type error protests that there is no instance of ^Constants^ for
^Encoding^ or ^String^. These instances are required because ^constants^ is
applied to the entire structure of ^Exp3^, including the representation of
^Const3^. Unfortunately, the Haskell type system cannot express that ^e^ will
never be constructed with ^Const3^ and that the offending instances would never
actually be invoked at run-time.

Indeed, the standard ^Constants^ semantics for ^Encoding^ and ^String^,
independent of their role in ^Exp3^, would be ^constants _ = []^ since they
contain no expression constants. This would render the ^Constants Exp3^
instance semantically incorrect. Given that this function is being defined
generically only to cope with the hypothetically numerous constructors for
arithmetic operators, for all of which the generic definition of ^constants^
suffices, it is dubious and perhaps even misleading to instantiate ^Constants^
at these two types at all. Moreover, if such instances had been declared and a
new constructor were added to ^Exp3^ that contained a ^String^ or an
^Encoding^, that constructor would silently adopt the generic definition of
^Constants^. Therefore, declaring instances of generic classes that are known
to be ultimately unnecessary is considered harmful. \todo{Similar to these
  required-but-extraneous instances?
  \url{http://hackage.haskell.org/trac/ghc/ticket/5499}}

An alternative instance, still expressable within \ig, avoids generating the
offending constraints on ^Encoding^ and ^String^ by manipulating the
sum-of-products representation directly.

\begin{haskell}
 -- NB speculative: valid, but obfuscated & immodular
instance Constants Exp3 where
  constants e = case from e of
    L (C (Var enc :*: Var s)) -> [decode enc s]
    R x -> constants x
\end{haskell}

\noindent Though this instance is well-typed and semantically correct, it is
also severely obfuscated and immodular, because it exposes the encoding of
^Exp3^'s structure. In particular, it assumes that ^Const3^ is the left summand
of ^Exp3^'s representation. Matters only worsen if the particular structural
encoding happens to bury the constructor(s) of interest deeper in the nestings
of ^:+:^. This definition is especially fragile with respect to changes of the
declaration of ^Exp3^, because the sum is often times expressed as a balanced
nesting of ^:+:^s (for compile-time efficiency); adding a new constructor, no
matter its position in the list of constructors is likely to displace ^Const3^
from the left summand. The definition of ^constants^ is ultimately so
obfuscated because the anonymous product representing ^Const3^ is used
directly, making no indication that ^Const3^ is the constructor of interest. We
consider these detriments to the software quality unacceptable.

Using a slight simplification of the \yoko\ API, the preferred instance is
declared as follows. Since the representation of the type's structure is not
exposed, this instance is precisely as modular as the ill-typed but ideal first
attempt. Furthermore, if the ^disband^, ^project^, and ^reps^ functions and the
^*_^ naming convention are recognized as pieces of a familiar library API, then
this instance is also nearly as lucid as the first: the special treatment of
^Const3^ is obvious.

\begin{haskell}
-- derived from Exp3
data Const3_ = Const3_ Encoding String
 -- a simplification of the #\yoko# API
goSolo :: Project a sum => sum -> Either a (sum :-: a)

instance Constants Exp3 where
  constants e = case project (disband e) of
    Left (Const3_ enc s) -> [decode enc s]
    Right x -> constants (reps x)
\end{haskell}

\noindent The ^Const3_^ constructor is the sole constructor of a type derived
by \yoko\ from the declaration of the ^Const^ constructor. We designate such
types \emph{fields types}. The ^disband^ function converts a data type to a
nested ^:+:^ sum of its constructors, each represented as a fields type---hence
the \emph{sum-of-constructors} generic view.

The ^project^ function, ^Project^ type class, and ^:-:^ type family are
operations on nested ^:+:^ sums. The ^project^ function uses ^:-:^ to remove a
type from a sum and determine whether a value in the sum was a value of the
removed type or part of the remaining sum, as computed by ^:-:^. The ^reps^
function completes the sum-of-products structural representation by converting
a sum of constructors to a sum of the corresponding products.

The ^Const3_^ type and the ^:-:^ family are the essential components that avoid
the generation of the superfluous ^Constants^ constraints on the ^Encoding^ and
^String^ types. In particular, the type of ^x^ in the ^Right^ branch is
^Rep___Exp3___:-:___Const3_^, which expresses to the typechecker the informal
notion of an ^Exp3^ value not constructed by ^Const3^. The remaining
constructors of ^Exp3^ do not incur the offending constraints when delegated to
the generic definition of ^constants^. This instance declaration thereby uses
^project^ as a form of pattern matching with a more sophisticated type rule
than the Haskell ^case^ expression's. The type of ^project^ locally refines
types in the branches in much the same manner as does supercompilation
\citep{supercompilation}. By delaying the representation of each constructor as
an anonymous product, the sum-of-constructor view grants this style of pattern
matching both robustness against ambiguities and the lucidity of genuine
constructor syntax.

% Metaprogramming with Template Haskell provides an alternative means to
% overcome the immodularity, by transforming a slight obfuscation of the ideal
% instance into the immodular one \citep{hackage-th-gd}.

\todo{Make sure to explain that the Par1 and Par2 types are because they allow Rec to
  used exactly.}

The capability of refining a set of types by projecting out certain
constructors is crucial to our goal of automating transformation functions. The
interesting constructors with semantics pertaining to the semantics of the
transformation cannot be mapped automatically. Thus, without a way to remove
them from the generic representation of a data type, no transformation can be
automated. The ^project^ and ^partition^ functions therefore underly automated
transformation by enabling the user to handle the constructors that
\yoko\ cannot transform automatically.

The simplicity of the strict sum-of-products generic view, \ie\ excluding the
^C^ type, precludes any reflection capabilities requiring non-structural
information. The \ig\ library extends sums and products with the ^C^ type in
order to enable generic definitions of classes like ^Show^ and ^Read^, which
necessarily involve the names of constructors. We consider \yoko's fields types
to be an evolution of the ^C^ type. In fact, the ^C^ constructor, and its
associated constructor types, could underly a functionality similar to the
^project^ function from the \yoko\ instance for ^Constants Exp3^. Thus, it is
possible to use the ^C^ type instead of fields types to preserve the modularity
of that instance declaration. However, without relying on Template Haskell
metaprogramming, the obfuscation of this non-modular \ig\ instance would
actually be more severe than that of the immodular \ig\ instance from the
previous section; some sort of type ascription is necessary in order to tag the
structural pattern with the desired constructor name. Fields types are a
natural solution that embraces the delayed representation hinted at by the ^C^
representation type's constructor types without involving heavyweight
metaprogramming.

\subsection{Reflection of Constructor Names}
\label{sec:reflecting-names}

\todo{We introduced fields types first because it is convenient to index the
  \inlineHaskell{Tag} instances with fields types. However, \inlineHaskell{Tag}
  instances can also be indexed by the first parameter of the \inlineHaskell{C}
  representation type from the \ig\ generic view: \inlineHaskell{Tag} is
  orthogonal to fields types.}

For the constructors that can be automatically transformed, the
sum-of-constructors view still does not reflect enough information. Beyond the
finer grained invocation of the generic definition on a per-constructor basis
demonstrated in the previous section, an improved reflection of constructors in
\yoko\ is also key to its support for generic definition of transformations
between types. The strict sum-of-products view distinguishes such constructors
only by their position in the sum. Reflection of non-structural information is
required in order to individually distinguish multiple constructors within a
type that share the same products structure. For example, the
^Plus___Exp___Exp^ and ^Mult___Exp___Exp^ constructors both have the same
structure, ^Rec Exp :*: Rec Exp^, and hence require more information to be
distinguished outside of a fixed sum. \yoko\ instead infers the correspondence
of constructors the same way users do: by the constructor name.

The library API includes a type family ^Tag^ for mapping a fields type to a
reflection of its constructor name as a type-level string. In the context of a
compiler pipeline, we suppose it is safe to rely on constructor names being
similar in adjacent data types; it is highly likely that in all of the types
the pipeline, each constructor corresponding to, say, applications will be
similarly named. It might even be the same name if the types are declared in
separate modules. If not, it is likely the same root same, say ^App^, with some
additional suffix.

\todo{For this presentation, write a \inlineHaskell{Letter} data type with a
  constructor per character that is allowed to occur in a Haskell
  identifier. Define the quadratic instances for the
  \inlineHaskell{EqualLetter} type family. Have \inlineHaskell{Tag} yield a
  promoted \inlineHaskell{[Letter]}.

\begin{haskell}
-- #\yoko# API
type family Tag dc :: [Letter]

type instance Tag Const_  = [LC, Lo, Ln, Ls, Lt]
type instance Tag Plus_   = [LP, Ll, Lu, Ls]
type instance Tag Const2_ = [LC, Lo, Ln, Ls, Lt, L2]
type instance Tag Plus2_  = [LP, Ll, Lu, Ls,     L2]
type instance Tag Const3_ = [LC, Lo, Ln, Ls, Lt, L3]
type instance Tag Plus3_  = [LP, Ll, Lu, Ls,     L3]
\end{haskell}

The generic transformation we define in
\tref{Section}{sec:polymorphic-generic-conversion} converts a sum of
constructors to a type under the assumption that each constructor in the sum
occurs in the type with the same name and a the same fields (modulo the
corresponding substitution of recursive occurrences). This transformation will
therefore be able to identify the corresponding cosntructors when converting
from the ^AnonTerm^ type to the ^TopTerm^ type (from
\tref{Section}{sec:objective}).

A generic transformation of constructors from the ^Exp^ type to the ^Exp2^ or
^Exp3^ type would require a more sophisticated type-level predicate on
strings. Since we are already emulating type-level strings, we only handle the
simpler case of types with constructors with the same name.

Reflection of a data type's set of constructors as well as the name and
structure of each constructor provides an abundance of information for
type-level programming in \yoko. The current Haskell features supporting such
type-level programming are powerful but rudimentary, and so the next two
sections introduce some fundamental machinery. The remaining technical sections
present the full \yoko\ API, define the lambda lifting transformation between
the ^AnonTerm^ and ^TopProg^ types, and then factor out its generic essence
into a generic transformer that can be directly reused for other
transformations.

\todo{subsection for new representation type grammar}

\section{Sets as Sums}
\label{sec:sets-as-sums}

\begin{figure}[h]
\begin{haskell}
-- #\tref{Section}{sec:sets-as-sums}#, Sets as sums
data Void                  -- empty set
newtype N a = N a          -- singleton set
data a :+: b = L a | R b   -- set union

-- #\tref{Section}{sec:embed-partition}#, Set operations
type family (:-:) sum sum2

class Embed sub sup where embed :: sub -> sup

inject :: Embed (N a) sum => a -> sum

class Partition sup sub where
  partition :: sup -> Either sub (sup :-: sub)

project ::
  Partition sum (N a) => sum -> Either a (sum :-: N a)
\end{haskell}
\caption{The API for sets as sums.}
\label{fig:sets-as-sums}
\end{figure}

The \yoko\ library reflects the set of constructors of a data type as a sum of
the corresponding fields types. \tref{Figure}{fig:sets-as-sums} on
page~\pageref*{fig:sets-as-sums} lists the API for sets of types in \yoko. Sets
of types are modeled as sums using nestings of the ^:+:^ type with set elements
delimited by applications of the ^N^ type, which models singleton sets. As an
example, the set of the base types ^Int^, ^Char^, ^Bool^, and ^()^ is
represented by the type ^Bases^, where
^type___Bases___=___(N___Int___:+:___N___Char)^ ^:+:^
^(N___Bool___:+:___N___())^.

Without the ^N^ type, certain essential type families cannot be indexed by type
sets, since this would require overlapping type family instances, which are
forbidden by Haskell's type system. The ^C^ representation type from \ig\ could
be used to model singleton sets, but the semantics of its two type parameters
are incompatible with field types.

\subsection{Embedding and Partitioning Sets}
\label{sec:embed-partition}

A value of any type can be injected into a sum modeling a set that contains
that type via the ^inject^ function. For example, any ^Int^, character,
Boolean, or unit can be injected into ^Bases^. A value of any subset of those
types, such as ^N Int :+: N Bool^ can be embedded into ^Bases^ via the ^embed^
function. Similarly, an element type can be projected via the ^project^
function and a set can be partitioned into two sets via the ^partition^
function.

The ^partition^ function uses the ^Either^ type to model the fact that a value
of the ^sup^ type may be an injection of a type not in the ^sub^ type. In that
case, the ^sup^ type is refined to the difference ^sup :-: sub^ since the
evaluation of the ^project^ function has established that its argument is not
an injection of one of the ^sub^ types. The ^:-:^ type is a simple type-level
function defined using two other type families.

\begin{haskell}
type instance (:-:) (N x) sum2 =
  If (Elem x sum2) Void (N x)
type instance (:-:) (l :+: r) sum2 =
  Combine (l :-: sum2) (r :-: sum2)
\end{haskell}

\noindent The ^Elem^ family implements the obvious membership predicate; it is
further discussed below. The ^Combine^ family maintains the invariant that the
^Void^ type does not occur as an summand within an otherwise non-empty set; for
example, ^N Int :+: Void^ violates this property.

\begin{haskell}
type family Combine sum sum2

type instance Combine Void x = x
type instance Combine (N x) Void = N x
type instance Combine (N x) (N y) = N x :+: N y 
type instance Combine (N x) (l :+: r) =
  N x :+: (l :+: r)
type instance Combine (l :+: r) Void = l :+: r
type instance Combine (l :+: r) (N y) =
  (l :+: r) :+: N y
type instance Combine (ll :+: rl) (lr :+: rr) =
  (ll :+: rl) :+: (lr :+: rr)
\end{haskell}

We choose to maintain this invariant here because the assumption that every
summand is non-degenerate simplifies the definition of the other type set
operations. Because \yoko\ uses type sets to reflect constructors and mutually
recursive types, empty sets are non-sensical. A data type with no constructors
only permits degenerate method definitions, which hardly require
genericity. Similarly, an empty family of mutually recursive types is no type
at all. Thus, any occurrence of ^Void^ outside of intermediate type-level
computations is considered user error. Hence the invariant.

The ^Elem^ family and the instances of ^Embed^ and ^Partition^ are ultimately
predicated upon the ^Locate^ family, which is not exposed to the user. Given a
target type and a sum, ^Locate^ calculates a path through the nested ^:*:^
types of the sum to reach the occurrence of the target type. Even though such a
path might not exist, ^Locate^ has a total defition by structuring its result
in a promotion of the ^Maybe^ type. The ^Elem^ type function simply checks if
its first argument can be located in its second.

\begin{haskell}
data Here a   ;   data Left x   ;   data Right x

type family Locate a sum

type instance Locate a Void = Nothing
type instance Locate a (N x) =
  If (Equal x a) (Just (Here a)) Nothing
type instance Locate a (l :+: r) =
  MaybeMap (Left r) (Locate a l) `MaybePlus1`
  MaybeMap (Right l) (Locate a r)

type Elem a sum = IsJust (Locate a sum)
\end{haskell}

The ^Here^, ^Left^, and ^Right^ types are type-level values, composed to model
a path through nested ^:*:^ types. The ^Equal^ type family is explained in
\tref{Section}{sec:scaffolding}. The omitted definitions of ^IsJust^,
^MapMaybe^, and ^MaybePlus1^ are obvious, excepting the caveat that the
^MaybePlus1^ family has no instance where both of its indices are constructed
with ^Just^. As a result, a located type cannot occur in the sum model of type
set more than once. In the context of these sets as constructors and sibling
types, this would never happen and would again indicate user error. The
specific semantics of ^MaybePlus1^ grant \yoko\ robustness against this sort of
ambiguity.

We omit the instances of ^Embed^ and ^Partition^. Both are defined via
auxiliary classes that branch according to the results of ^Locate^, and hence
accomodate no ambiguity. This implementation technique avoids the overlapping
instances Haskell language extension, which would require sums to be linearized
in order to define the instances. Our presumption of a type-level type-equality
predicate (explained in \tref{Section}{sec:scaffolding}) is a linchpin of this
technique. The elided instances of ^Embed^ and ^Partition^ include any type set
built with ^N^ and ^:+:^; ^Void^ is not supported.

\section{The \yoko\ Generic View}
\label{sec:yoko-api}

\tref{Figure}{fig:yoko-api} on page~\pageref*{fig:yoko-api} lists the core
declarations of the \yoko\ generic view. We demonstrate its application with a
working example that leads into the lambda lifting example. Without loss of
generality, we concretize the ^AnonTerm^'s underspecified set of non-nominal
constructors. There will be a single non-nominal constructor, ^App^, modelling
application. Working through a concrete type is easer to follow, but the ^App^
constructor is still intended to exemplify an unlimited number of other
non-nominal constructors, all of which would be handled by the lambda lifting
definition as developed in the next section.

\begin{haskell}
module AnonymousFunctions where
  data AnonTerm = Lam Type AnonTerm | Var Int
                | Let [Decl] AnonTerm
                | App AnonTerm AnonTerm   -- new
  newtype Decl = Decl Type AnonTerm       -- no change
\end{haskell}

Each of the following subsections introduce a part of the \yoko\ view and
instantiates it for the ^AnonTerm^ and ^Decl^ types. The presentation proceeds
bottom up, discussing first the representation of fields, then constructors,
and finally whole data types.

\begin{figure}[h]
\begin{haskell}
-- #\tref{Section}{sec:field-representation}#, Representation of fields
data U       = U         -- empty tuple
data a :*: b = a :*: b   -- tuple concatenation

data Dep a = Dep a
data Rec t = Rec t

data Par1 f a    = Par1 (f a)
data Par2 ff a b = Par2 (ff a b)

type family Rep dc
class Generic dc where
  rep :: dc -> Rep dc
  obj :: Rep dc -> dc

-- #\tref{Section}{sec:reflect-dcs}#, Reflection of constructors
type family Tag dc :: String -- see footnote#\footnotemark#

type family Range dc
class (Generic dc, DT (Range dc),
       Embed (N dc) (DCs (Range dc))) =>
  DC dc where rejoin :: dc -> Range dc

-- #\tref{Section}{sec:reflect-dts}#, Reflection of data types
newtype DCsOf t sum = DCsOf sum

type family DCs t
type Disbanded t = DCsOf t (DCs t)

class DT t where disband :: t -> Disbanded t

goSolo ::
  (DT (Range dc), Partition sum (N dc)) =>
  DCsOf (Range dc) sum ->
  Either dc (DCsOf (Range dc) (sum :- N dc))
\end{haskell}
\caption{The \yoko\ generic view.}
\label{fig:yoko-api}
\end{figure}

\footnotetext{We assume type-level strings for this presentation. The actual
  implementation emulates type-level strings as in
  \tref{Appendix}{sec:scaffolding}.}

\subsection{Representation of Fields}
\label{sec:field-representation}

The delayed representation of \yoko\ expands the role of \ig's constructor
types. Each fields type represents a constructor independently of its data type
of origin. Reflection of the ^AnonTerm^ and ^Decl^ types uses the five obvious
fields types.

\begin{haskell}
data Lam_  = Lam_  Type AnonTerm
data Var_  = Var_  Int
data Let_  = Let_  [Decl] AnonTerm
data App_  = App_  AnonTerm AnonTerm
data Decl_ = Decl_ Type AnonTerm
\end{haskell}

As an extension of the \ig\ generic view, the \yoko\ generic view largely
inherits its representation types. The basic structural representation types
remain the same, excepting the addition of ^N^ and the renaming of ^Var^ to
^Dep^. The representation types also include the new ^Par1^ and ^Par2^ types
for the navigation of compound fields. Finally, the ^C^ type is removed in
favor of fields types.

As an extension of the \ig\ approach, the \yoko\ approach centers around the
^Rep^ type family. The ^Rep^ family in \yoko\ maps constructor types, as
opposed to entire data types in the \ig\ approach, to their structural
representation. Accordingly, sums do not occur in \yoko\ ^Rep^ instances. In
combination with operations on sets of fields types, this allows more granular
invocation generic definitions as demonstrated in
\tref{Section}{sec:granularity}. The constructors of ^AnonTerm^ and ^Decl^ are
represented as follows.

\begin{haskell}
type instance Rep Lam_  = Dep Type :*: Rec AnonTerm
type instance Rep Var_  = Dep Int
type instance Rep Let_  =
  Par1 [] (Rec Decl) :*: Rec AnonTerm
type instance Rep App_  = Rec AnonTerm :*: Rec AnonTerm
type instance Rep Decl_ = Dep Type     :*: Rec AnonTerm
\end{haskell}

\noindent The \ig\ approach has no difficulty with mutual recursion, and the
^Par1^ representation type from \tref{Section}{sec:compound-fields} handles the
compound first field of ^Let^.

The methods of the ^Generic^ class implement the isomorphism between a fields
type and its representation. This class has the same semantics as the
^Representable^ class from \ig. The only distinction is that sums are absent,
since they are never present in the righthand side of ^Rep^
instances. Instances of ^Generic^ follow directly from the corresponding ^Rep^
instances, so we omit them for ^AnonTerm^ and ^Decl^.

\subsection{Reflection of Constructors}
\label{sec:reflect-dcs}

The \yoko\ reflection of constructors is more thorough than the ^Constructor^
class of the \ig\ library. The major enhancements are the software quality
benefits of fields types and the type-level programming enabled by the ^Tag^
type family, as discussed in \tref{Section}{sec:reflecting-names}.

The ^Tag^ family maps a fields type to a type-level reification of the
constructor's name. Generic transformation between two similar types relies
essentially on the reflection of constructor names independently of the type of
origin.

\begin{haskell}
type Tag Var_  = [LV, La, Lr]
type Tag Lam_  = [LL, La, Lm]
type Tag Let_  = [LL, Le, Lt]
type Tag App_  = [LA, Lp, Lp]
type Tag Decl_ = [LD, Le, Lc, Ll]
\end{haskell}

In order to relate a fields type back to its type of origin, \yoko\ declares
the ^Range^ type family. The ^Range^ family pervades the reflective API,
connecting fields types back to their original type.

\begin{haskell}
type Range Lam_  = AnonTerm
type Range Var_  = AnonTerm
type Range Let_  = AnonTerm
type Range App_  = AnonTerm
type Range Decl_ = Decl
\end{haskell}

A type ^dc^ is a genuine fields type only if it is an instance of the ^DC^
class. The super class constraints require that ^dc^ can be converted to a
sum-of-products view, that a ^dc^ value can be injected into its range, and
that its range is a data type satisfying the ^DT^ class, discussed in the next
subsection. The ^rejoin^ method specifies the correspondence between ^dc^ and
its twin constructor in its range type. We do not retain the ^occName^ method
from the \ig\ ^Constructor^ class, since the rest of our developments do not
rely on it. Moreover, with proper support for type-level strings, it could be
defined as the \emph{demotion} of the ^Tag___dc^ type-level string.

\begin{haskell}
instance DC Lam_  where
  rejoin (Lam_  ty  tm ) = Lam ty tm
instance DC Var_  where
  rejoin (Var_  i      ) = Var i
instance DC Lam_  where
  rejoin (Let_  ds  tm ) = Let ds tm
instance DC App_  where
  rejoin (App_  tm1 tm2) = App tm1 tm2
instance DC Decl_ where
  rejoin (Decl_ ty tm  ) = Decl ty tm
\end{haskell}

\subsection{Reflection of Data Types}
\label{sec:reflect-dts}

The only distinction between reflection of data types in \ig\ and in \yoko\ is
the delayed representation of constructors. Instead of mapping an entire data
type to its sum-of-products view in one step, \yoko\ maps a data type to a sum
of its fields types. Since \yoko\ reserves the ^Rep^ family for mapping a
fields type to its structural representation, it declares the new ^DCs^ type
family for specifying the representation of a data type.

\begin{haskell}
type instance DCs AnonTerm =
  (N Lam_ :+: N Var_) :+: (N Let_ :+: N App_)
type instance DCs Decl = N Decl_
\end{haskell}

The phantom type parameter of the ^DCsOf^ type tags a sum of constructors with
their shared range type. This declaration itself does not enforce those
semantics, but other functions in \yoko\ use the ^DCsOf^ type under the
assumption of these semantics, sometimes enforcing them. For ^*^ data types,
this rarely provides more than robustness against user error by preventing the
unintended mixing of fields types with disparate ranges within the same the
sum. For higher-kinded types, however, it provides an avenue for inference to
unify the type parameters of polymorphic values, thereby mitigating frequent
ascriptions.

For example, the \yoko\ reflection of lists involves the fields type ^Nil_^, in
which the type parameter is phantom.

\begin{haskell}
data Nil_ a = Nil_
type instance Range (Nil_ a) = [a]
\end{haskell}

\noindent Since the type-level programming both within \yoko\ as well as
involved in typical uses of \yoko\ relies on type-equality, the polymorphism of
^Nil_^ would often cause an unfortunate avalanche of type errors, were it not
for the ^DCsOf^ type. While the user could ascribe the intended type parameter,
it is oftentimes burdensome to do so on account of scoped type
variables. However, the ubiquity of the ^DCsOf^ type throughout the API
mitigates this burden. For example, if the ^null_^ function is defined as
follows, the inferred type is ^[a] -> Bool^.

\begin{haskell}
-- null_ :: [a] -> Bool   -- inferred via DCsOf
null_ x = case goSolo (disband x) of
  Left Nil_ -> True
  _ -> False
\end{haskell}

If the ^DCsOf^ wrapper were hypothetically not maintained by the ^disband^ and
^goSolo^ functions, the resulting inferred type of ^null_^ would actually
render the function inapplicable\footnote{Indeed, GHC generates an type error
  with the suggestion: ``Probable cause: the inferred type is
  ambiguous''.}. Its inferred type would involve two separate and completely
unrelated type variables: one for the type parameter of the ^Nil_^ pattern and
another for the type of ^null_^'s argument.

\begin{haskell}
-- without DCsOf,
--   null_ :: (DT a, Partition (DCs a) (N (Nil_ b))) =>
--           a -> Bool
\end{haskell}

In particular, ^Nil_^'s type parameter would be unreachable in the type of
^null_^. The type variable would, however, participate in a ^Partition^
constraint, rendering it unsatisfiable. This scenario is avoided because the
types of ^goSolo^ and ^disband^ do use the ^DCsOf^ wrapper to unify the shared
range type of their arguments. Via ^DCsOf^ tag, the type of ^x^ is unified with
^Range (Nil_ a)^, \ie\ ^[a]^. The type parameter of the ^Nil_^ pattern is thus
propogated through the types of the \yoko\ API elements all the way to the
argument of ^null_^.

The ^Disbanded^ type synonym wraps the ^DCsOf^ tag around a type's sum of
constructors representation. This synonym occurs in the type of the ^DT^ type
class's ^disband^ method, which specifies how to disband the constructors of a
data type by converting each constructor to a value in the corresponding sum of
constructors.

\begin{haskell}
instance DT AnonTerm where
  disband (Lam ty tm)   = inject $ Lam_ ty tm
  disband (Var i)       = inject $ Var_ i
  disband (Let ds tm)   = inject $ Let_ ds tm
  disband (App tm1 tm2) = inject $ App_ tm1 tm2
instance DT Decl where
  disband (Decl ds tm) = inject $ Decl_ ds tm
\end{haskell}%$

\noindent The ^disband^ method complements the ^rejoin^ method of the ^DC^
class, akin to the inverse relationship of the ^rep^ and ^obj^ methods of the
^Generic^ class.

\subsection{Discussion}

Once the concept of fields types is understood, all of these instances are
intuitive. They are also straight-forward to generate, and even fields types
can be generated.

In the following, we assume that ^TopTerm^ and all of its constructors are
completely reflected in the \yoko\ generic view.

\section{The Lambda Lifting Transformation}
\label{sec:lambda-lift-definition}

\stub

\subsection{Monomorphic}

\begin{figure}[p]
\begin{haskell}
type family FindDCs s sum
type instance FindDCs s (N dc) =
  If (Equal s (Tag dc)) (Just (N dc)) Nothing
type instance FindDCs s (a :+: b) =
  DistMaybePlus (FindDCs s a) (FindDCs s b)

-- TODO say something smart here
class ToExp2 a where toExp2 :: a -> Exp2

instance ToExp2 sum => ToExp2 (DCsOf t sum) where
  toExp2 = toExp2 . unDCsOf

instance ToExp2 Exp where
  toExp2 e = case goSolo (disband e) of
    Left  (Const_ i) -> Const2 i Nothing
    Right x          -> toExp2 x

-- mapping over the sum
instance (ToExp2 a, ToExp2 b) => ToExp2 (a :+: b) where
  toExp2 = foldPlus toExp2 toExp2

instance (Generic dc,
          Just (N dc') ~ FindDCs (Tag dc) (DCs Exp2),
          ToExp2Rs (Rep dc),
          Rep dc' ~ RsToExp2 (Rep dc),
          DC dc', Range dc' ~ Exp2) =>
  ToExp2 (N dc) where
  toExp2 (N x) = rejoin ((obj . toExp2Rs . rep) x :: dc')

-- mapping into just the recursive fields
type family RsToExp2 prod
class ToExp2Rs prod where toExp2Rs :: prod -> RsToExp2 prod

type instance RsToExp2 (R a) = R Exp2
instance ToExp2 a => ToExp2Rs (R a) where
  toExp2Rs (R x) = R (toExp2 x)

type instance RsToExp2 (D a) = D a
instance ToExp2Rs (D a) where toExp2Rs = id

type instance RsToExp2 (a :*: b) =
  RsToExp2 a :*: RsToExp2 b
instance (ToExp2Rs a, ToExp2Rs b) =>
  ToExp2Rs (a :*: b) where
  toExp2Rs = mapTimes toExp2Rs toExp2Rs
\end{haskell}
\caption{The lambda lifting transformation.\label{fig:dc-conversion}}
\end{figure}

We have not declared instances for ^U^, ^Par1^, nor ^Par2^ because they are not
required for this lambda lifting example. Since no constructors in the involved
types have zero fields, ^U^ never occurs. In the instance of ^ToExp2^, the
^Let^ constructor is not delegated to the generic definition, so its compound
field does not generate a ^ToExp2^ constraint involving ^Par1^.

\subsection{Polymorphic}
\label{sec:polymorphic-generic-conversion}

\begin{figure}[h]
\begin{haskell}
type family Cnv cnv a
class Convert cnv a where
  convert :: cnv -> a -> Cnv cnv a

type instance Cnv cnv (DCsOf t sum) = Cnv cnv t
instance ConvertTo cnv sum (Cnv cnv t) =>
  Convert cnv (DCsOf t sum) where
  convert cnv = convertTo cnv . unDCsOf

-- mapping over the sum
class ConvertTo cnv sum t where
  convertTo :: cnv -> sum -> t

instance (ConvertTo cnv a t, ConvertTo cnv b t) =>
  ConvertTo cnv (a :+: b) t where
  convertTo cnv =
    foldPlus (convertTo cnv) (convertTo cnv)

instance (Generic dc,
          Just (N dc') ~ FindDCs (Tag dc) (DCs t),
          ConvertRs cnv (Rep dc),
          Rep dc' ~ RsConvert cnv (Rep dc),
          DC dc', Range dc' ~ t, DT t) =>
  ConvertTo cnv (N dc) t where
  convertTo cnv (N x) =
    rejoin ((obj . convertRs cnv . rep) x :: dc')

-- mapping into just the recursive fields
type family RsConvert cnv prod
class ConvertRs cnv prod where
  convertRs :: cnv -> prod -> RsConvert cnv prod

type instance RsConvert cnv (R a) = R (Cnv cnv a)
instance Convert cnv a => ConvertRs cnv (R a) where
  convertRs cnv (R x) = R (convert cnv x)

type instance RsConvert cnv (D a) = D a
instance ConvertRs cnv (D a) where convertRs _ = id

type instance RsConvert cnv (a :*: b) =
  RsConvert cnv a :*: RsConvert cnv b
instance (ConvertRs cnv a, ConvertRs cnv b) =>
  ConvertRs cnv (a :*: b) where
  convertRs cnv =
    mapTimes (convertRs cnv) (convertRs cnv)
\end{haskell}
\caption{Generic constructor conversion.\label{fig:generic-dc-conversion}}
\end{figure}

\begin{haskell}
data ToExp2 = ToExp2
type instance Cnv ToExp2 Exp = Exp2

instance Convert ToExp2 Exp where
  convert cnv e = case goSolo (disband e) of
    Left  (Const_ i) -> Const2 i Nothing
    Right x          -> convert cnv x
\end{haskell}

Type-level programming is a well-known and powerful capability of Haskell, but
such programming unfortunately has limited specific support in the language. In
particular, type-level strings and a type-level equality are features planned
for the immediate future. For this presentation, we will presume they exist; we
discuss our emulation in \tref{Appendix}{sec:scaffolding}.

\section{Discussion}

\stub

\section{Related Work}

\stub

\section{Conclusion}

\stub

\subsection{Fields Types}

We developed the \yoko\ library by exploring the concepts underlying this
^Constants^ instance as the foundation of a generic programming library. In
particular, the decoupling of constructors from their data types and sibling
constructors enables many of the library's enhancements. This decoupling is
most explicit in the novel fields types, as demonstrated by the use of
^Const3_^. Each fields type is a simple nominal type with a single constructor
emulating an eponymous constructor of the original data type.

The purpose of fields types is to make the benefits of a genuine constructor---
including concise syntax, familiarity, and direct admittance as a
pattern---more widely available. Thus a fields type is a copy of constructor
that is can be used in more contexts. In particular, fields types grant
totality to functions that match just a single constructor, which is the most
basic building block within the assembly of ad-hoc behaviors for a subset of
data type's constructors. The direct use of fields types in the \yoko\ generic
view prevents obfuscation and immodularity when exposing to the user values of
representations, which is natural in certain generic capabilities, especially
per-constructor overrides (as in ^Constants^), zippers, algebras, and other
type-indexed data types \citep{indexed-data}.

The fields of constructors are ultimately represented by the exact same
representation types as in \ig, via the ^reps^ function. However, delaying the
purely structural representation by relying on the fields types allows partial
exposure of representation---an occasional necessity---without incurring
drastic detriments to code quality, such as immodularity.

\appendix

\section{Scaffolding for Type-level Programming}
\label{sec:scaffolding}

\stub

The type-level programming involved in \yoko's most advanced capabilities
requires significant but quite fundamental scaffolding. This sections declares
these and other related mechanisms.

\subsection{Promotion and Proxies}

\stub

Explain the
\href{http://hackage.haskell.org/package/type-bool}{\inlineHaskell{type-bool}}
package.

Unfortunately, kind-indexed type families are not yet implemented. As a result,
using data kinds for some type families would be overly restrictive. For
example, the best currently expressible kind for ^IsJust^ is
^Maybe___*___->___Bool^. Because subkinding is not supported, ^IsJust^ cannot
be applied to ^Maybe Bool^ or ^Maybe (Maybe *)^. The ideal kind is
^forall___k.___Maybe___k___->___Bool^, but GHC currently has no syntax for
expressing that kind. Thus, we do not promote higher-kinded data to kinds. For
promoting data types of kind ^*^, there is no need for polymorphism, we do
promote ^Bool^.

\begin{