LambdaCube 3D API 0.5

We gradually improve the documentation. If you have any question or something is confusing and needs to be clarified please ask us via email or irc on #haskell-game (irc.freenode.net).

Reduction phases

Reduction is done in three different phases:

• Compile time CPU
• Runtime CPU
• Runtime GPU

The runtime CPU and runtime GPU phases are interleaved.

Design principles:

1. Every reduction step is done in the runtime CPU phase by default.
2. As an optimization, some reductions are done at runtime GPU instead of runtime CPU. Of course, this optimization step is crucial: without this optimization LambdaCube 3D is useless.
   GPU operations are very slow on CPU, but the CPU implementation of GPU primitives ensures that there is no cheating in the types of GPU primitives.
3. As an optimization, some reductions are done at compile time CPU instead of runtime CPU.
4. The programmer can assert that a computation is done in a specific phase via the type system.
   In this way, one can refactor LambdaCube 3D source code without fear of performance regressions.

Principle 1. is not yet implemented: some reductions are only possible to do at the runtime GPU phase.

Principle 4. is not implemented yet: phase information is provided in the comments, not in the types. The compiler code generator automagically determines the phases of expressions.

Basic types

Basic values

The difference between Word and Nat is that they have different representations. Word values are atomic, whilst Nat values have more structure: for example the pattern Succ a matches all non-zero naturals. Usually Nat is used in types, for example, the dimension of a Vec is a Nat instead of an Int.

The vector constructors are V2, V3 and V4. It is only possible to construct vectors with 2, 3 or 4 dimensions.
For example,

    V3 0 1.1 2 :: Vec 3 Float

The matrix constructors are M22F, M23F, M24F, M32F, M33F, M34F, M42F, M43F and M44F. The matrix constructors needs the columns of the matrices.
For example,

    M23F (V2 0 0.1) (V2 1 1) (V3 0 0) :: Mat 2 3 Float

It is only possible to construct matrices with dimensions i x j where i and j are 2, 3 or 4.
It is only possible to construct martices of float elements.

Basic type families (type functions)

VecScalar        :: Nat -> Type -> Type
MatVecScalarElem :: Type -> Type

If the first argument of VecScalar is 1, then the result is the second argument, otherwise VecScalar is the same as Vec.
Examples:

VecScalar 1 Float = Float
VecScalar 2 Float = Vec 2 Float

MatVecScalarElem gives back the type of elements of matrices and vectors; in case of scalars it is the identity function.
Examples:

MatVecScalarElem Float = Float
MatVecScalarElem (Vec 3 Bool) = Bool
MatVecScalarElem (Mat 3 3 Float) = Float

Basic type classes

Primitive operations

Phase information
Runtime CPU: None of the primitive oparations are accessible.
Runtime GPU: All of the primitive oparations are accessible.
Compile time CPU: Some of the primitive oparations are accessible.

Values

rgb :: Float -> Float -> Float -> Vec 4 Float
rgb r g b = V4 r g b 1.0

Transformation Functions

Arithmetic Functions

The functions operate component-wise.

Geometric Functions

Matrix Functions

Vector and Scalar Relational Functions (component-wise)

Logic Functions

Angle and Trigonometry Functions

The functions operate component-wise.

Exponential Functions

The functions operate component-wise.

Common Functions

Fragment Processing Functions

Noise Functions

Bit-wise Functions

The functions operate component-wise and bit-wise.

The functions operate component-wise.

Integer/Float Conversion Functions

The functions operate component-wise.

Common data structures

Homogeneous and heterogeneous lists (lists and tuples)

See the language specification

Maybe data type

Maybe has the standard definition:

data Maybe a = Nothing | Just a

Vector data type

data Vector (n :: Nat) t

A Vector n t is an vector with length n.
The difference between Vector and Vec is that there is no restriction on the length of a Vector.

Phase information
Currently Vector is an auxiliary phantom type used in the description of Fragment.

Special data structures

Geometric primitives

There are three type of primitives: triangles, lines and points.

data PrimitiveType
    = Triangle
    | Line
    | Point

A Primitive value is a container of 1, 2 or 3 values tagged with a PrimitiveType.

data Primitive a :: PrimitiveType -> Type where
    PrimPoint    :: a           -> Primitive a Point
    PrimLine     :: a -> a      -> Primitive a Line
    PrimTriangle :: a -> a -> a -> Primitive a Triangle

In LambdaCube3D, Primitive is always paramterized by a heterogeneous list whose elements are called attributes.

Primitive streams

PrimitiveStream is a central data type of GPUs:

type PrimitiveStream a p = [Primitive p a]

Mapping PrimitiveStreams

mapPrimitives :: (a -> b) -> PrimitiveStream p a -> PrimitiveStream p b
mapPrimitives f = map (mapPrimitive f)

mapPrimitive :: (a -> b) -> Primitive a p -> Primitive b p

You never need to call mapPrimitive directly; use mapPrimitives instead, which is compiled to runtime GPU code. (The compiled code will be part of the so-called vertex shader.)

Construction of PrimitiveStreams

There are two ways to construct a PrimitiveStream. The direct way is documented here, the other way is documented in Interface for inputs.

fetchArrays gets a heterogenous list of attribute lists. The atribute lists should have the same length.

fetchArrays :: HList x -> PrimitiveStream p (HList (map ListElem x))

type family ListElem a where ListElem [a] = a

Example usage:

    fetchArrays ([1.0, 2.0, 3.0], [True, False, True])
        :: PrimitiveStream Triangle (Float, Bool)

Restriction: If the primitive type is Triangle then the length of the lists should be divisible by 3; if the primitive type is Line then the length of the lists should be divisible by 2.
(The higher level API will detect misuse of fetchArrays statically.)

Fragments

A Fragment n t value is an n-vector of Maybe (SimpleFragment t) values. Maybe is used because simple fragments may be filtered out with the filterFragments function (see later).

type Fragment n t = Vector n (Maybe (SimpleFragment t))

A SimpleFragment value is an attribute tuple plus a viewport coordinate with depth value:

data SimpleFragment t = SimpleFragment
    { sFragmentCoords   :: Vec 3 Float  -- x, y, depth
    , sFragmentValue    :: t
    }

Basic functions on fragments:

customizeDepth :: (a -> Float) -> Fragment n a -> Fragment n a
filterFragment :: (a -> Bool)  -> Fragment n a -> Fragment n a
mapFragment    :: (a -> b)     -> Fragment n a -> Fragment n b

Fragment streams

FragmentStream is a list of fragments:

type FragmentStream n t  = [Fragment n t]

Operations on FragmentStreams:

customizeDepths :: (a -> Float) -> FragmentStream n a -> FragmentStream n a
customizeDepths f = map (customizeDepth f)

filterFragments :: (a -> Bool) -> (FragmentStream n a) -> (FragmentStream n a)
filterFragments p = map (filterFragment p)

mapFragments :: (a -> b) -> FragmentStream n a -> FragmentStream n b
mapFragments f = map (mapFragment f)

Phase information
customizeDepths, filterFragments and mapFragments are compiled to runtime GPU code.
The other operations are not accessible in any phase.

Images

Image kinds

There are different kind of images depending on the role the image plays:

data ImageKind
    = Color (c :: Type)     -- the image stores `c`-values
    | Depth                 -- the image stores depth values (`Float`s)
    | Stencil               -- the image stores stencil values (`Int`s)

The type of the stored values is calculated by imageType:

imageType :: ImageKind -> Type
imageType (Color c) = c
imageType Depth     = 'Float
imageType Stencil   = 'Int

Image type

An Image n t is n times a rectangle of imageType t-pixels:

type Image (n :: Nat) (t :: ImageKind) = Vector n [[imageType t]]

The n parameter is the so called layer count, which count the number of layers. Usually it is 1.

The layer count of an image:

type family ImageLC a :: Nat where ImageLC (Image n t) = n

Functions which construct clear images:

ColorImage   :: (Num t, color ~ VecScalar d t) => color  -> Image a (Color color)
DepthImage   :: Float  -> Image n Depth
StencilImage :: Int    -> Image n Stencil

Some wrapper functions for creating 1-layered clear images:

emptyDepthImage = DepthImage @1
emptyColorImage = ColorImage @1

Other image construction functions (texture support):

PrjImage            :: FrameBuffer 1 a -> Image 1 a
PrjImageColor       :: FrameBuffer 1 (Depth Float, Color (Vec 4 Float)) -> Image 1 (Color (Vec 4 Float))

Phase information
ColorImage, DepthImage, StencilImage, PrjImage and PrjImageColor are compiled to runtime GPU code.
The other operations are not accessible in any phase.

Framebuffers

A FrameBuffer n t is a tuple of images:

type FrameBuffer (n :: Nat) (t :: [ImageKind]) = HList (map (Image n) t)

Construction of framebuffers

One can construct a framebuffer from images if all have the same layer count:

imageFrame :: forall (a :: [Type]) . (sameLayerCounts a) => HList a -> FrameBuffer (ImageLC (head a)) (map GetImageKind a)

sameLayerCounts a = allSame (map 'ImageLC a)

allSame :: [a] -> Type
allSame [] = 'Unit
allSame [x] = 'Unit
allSame (x: y: xs) = 'T2 (x ~ y) (allSame (y:xs))

Rasterization

Rasterization turns a Primitive value into a list of Fragment values.
For this, a rasterization context is needed, and also information how interpolation should be done.

Rasterization contexts

There are different rasterization contexts for each primitive types:

data RasterContext a :: PrimitiveType -> Type where
    TriangleCtx  :: CullMode -> PolygonMode a -> PolygonOffset -> ProvokingVertex -> RasterContext a Triangle
    LineCtx      :: Float -> ProvokingVertex                                      -> RasterContext a Line
    PointCtx     :: PointSize a -> Float -> PointSpriteCoordOrigin                -> RasterContext a Point

Triangles can be discarded based on their apparent facing, a process known as Face Culling:

data CullMode
    = CullFront     -- triangle faces whose vertices appear counter-clockwise are culled
    | CullBack      -- triangle faces whose vertices appear clockwise are culled
    | CullNone      -- no culling

Triangles can be rendered in three ways according to PolygonMode values. Note that in WebGL triangles are allways filled regardless of PolygonMode values.

data PolygonMode a
    = PolygonFill                   -- filled triangles (mostly used)
    | PolygonPoint (PointSize a)    -- only triangle vertices are rendered as squares with the given size
    | PolygonLine Float             -- only triangle edges are rendered with the given width

PolygonOffset is used to eliminate the visual unpleasantness when you want to highlight the edges of a solid object. See glPolygonOffset.

data PolygonOffset
    = NoOffset              -- no offset
    | Offset Float Float    -- how to tweak depth values: *factor* and *units*

ProvokingVertex values are used only in case of Flat interpolation (see later).

data ProvokingVertex
    = LastVertex            -- the value of the last vertex is used in flat interpolation
    | FirstVertex           -- the value of the first vertex is used in flat interpolation

Points will be rendered as squares.
The square sides’ size is can given in two ways with PointSize data:

data PointSize a
    = PointSize Float                   -- static point size
    | ProgramPointSize (a -> Float)     -- dynamic point size which depends on attributes

Point are rendered as squares which can be texured. PointSpriteCoordOrigin tells where is the origin for the texure.

data PointSpriteCoordOrigin
    = LowerLeft       -- texture origin is lower left corner
    | UpperLeft       -- texture origin is lower left corner

Interpolation types

During rasterization, attribute values are interpolated between the vertices of triangles and lines.

OpenGL supports three types of interpolation:

data Interpolated t where
  Flat          ::                 Interpolated t   -- no interpolation
  Smooth        :: (Floating t) => Interpolated t   -- smooth interpolation
  NoPerspective :: (Floating t) => Interpolated t   -- no perspective correction

In case of Flat interpolation, either the value of the first vertex or the value of the last vertex is chosen. The ProvokingVertex value of the rasterization context tells which one to choose (see above).

The rasterizePrimitive function

rasterize turns a Primitive into a FragmentStream, given a rasterization contex and info for interpolation:

rasterizePrimitive
    :: a ~ Vec 4 Float: b
    => HList (map Interpolated b)                -- tuple of Smooth & Flat
    -> RasterContext (HList a) pr
    -> Primitive (HList a) pr
    -> FragmentStream 1 (HList b)

Notes:

• The first attribute of the primitive stream should have type Vec 4 Float.
  This attribute is omitted in the result fragment stream.
• One Interpolated value is needed for each fragment attribute for rasterization.

Rasterization of primitive streams

Always use rasterizePrimitives instead of rasterize becase the former is compiled to runtime GPU code:

rasterizePrimitives
    :: a ~ Vec 4 Float: b
    => HList (map Interpolated b)              -- tuple of Smooth & Flat
    -> RasterContext (HList a) pr
    -> PrimitiveStream pr (HList a)
    -> FragmentStream 1 (HList b)
rasterizePrimitives ctx is s = concat (map (rasterize is ctx) s)

Accumulation

Accumulation combines overlaying fragments. How to combine two overlaying fragments is described by an accumulation context.

The result of the accumulation is a framebuffer (a tuple of images).

Accumulation contexts

An accumulation context is a tuple which contains one fragment operation for each fragment attribute.

There are three kind of fragment operations depending of the image kind:

data FragmentOperation :: ImageKind -> Type where
  ColorOp   :: Num c
            => Blending c           -- blending function
            -> VecScalar d Bool     -- blending filter
            -> FragmentOperation (Color (VecScalar d c))
  DepthOp   :: ComparisonFunction
            -> Bool
            -> FragmentOperation Depth
  StencilOp :: StencilTests
            -> StencilOps
            -> StencilOps
            -> FragmentOperation Stencil

The ImageKind of a fragment operation:

type family FragmentOperationKind a :: ImageKind where
    FragmentOperationKind (FragmentOperation x) = x

Blending is combination of color values.

data Blending :: Type -> Type where
  NoBlending    ::                                   Blending t
  BlendLogicOp  :: (Integral t) => LogicOperation -> Blending t
  Blend         :: (BlendEquation, BlendEquation)
                -> ((BlendingFactor, BlendingFactor), (BlendingFactor, BlendingFactor))
                -> Vec 4 Float
                -> Blending Float

data LogicOperation
    = Clear
    | And
    | AndReverse
    | Copy
    | AndInverted
    | Noop
    | Xor
    | Or
    | Nor
    | Equiv
    | Invert
    | OrReverse
    | CopyInverted
    | OrInverted
    | Nand
    | Set

data BlendingFactor
    = ZeroBF
    | OneBF
    | SrcColor
    | OneMinusSrcColor
    | DstColor
    | OneMinusDstColor
    | SrcAlpha
    | OneMinusSrcAlpha
    | DstAlpha
    | OneMinusDstAlpha
    | ConstantColor
    | OneMinusConstantColor
    | ConstantAlpha
    | OneMinusConstantAlpha
    | SrcAlphaSaturate

data BlendEquation
    = FuncAdd
    | FuncSubtract
    | FuncReverseSubtract
    | Min
    | Max

Comparison functions for depth values.

data ComparisonFunction
    = Never
    | Less
    | Equal
    | Lequal
    | Greater
    | Notequal
    | Gequal
    | Always

Stencil operations.

data StencilOperation
    = OpZero
    | OpKeep
    | OpReplace
    | OpIncr
    | OpIncrWrap
    | OpDecr
    | OpDecrWrap
    | OpInvert

Accumulation function

The overlay function overlays the fragments of a fragment streams on the given background images:

overlay
    :: forall (n :: Nat) (c :: [Type])
    . b ~ map FragmentOperationKind c
    => FrameBuffer n b                            -- backround images
    -> ( HList c                                  -- fragment operations (accumulation context)
       , FragmentStream n (HList (imageType' b))  -- fragment steam
       )
    -> FrameBuffer n b                            -- result images

imageType' :: [ImageKind] -> [Type]
imageType' (Depth: x) = map imageType x 
imageType' x = map imageType x 

Notes:

• No fragment attribute is needed for overlaying depth images: the depth value is taken from the fragment viewport coordinates.

accumulateWith just pairs an accumulation context with a fragment stream:

accumulateWith ctx x = (ctx, x)

Getting inputs

The input of a pipeline consists of primitive streams, uniforms and textures.

Uniforms and textures

Uniforms can be imported by their name:

Uniform   :: String -> t

Note that currently the type of the uniform is not statically checked.

A texture is a uniform with type Texture. Currently textures are treated specially in LambdaCube 3D but this is planned to be changed.

Primitive streams

A primitive stream can be imported by fetch:

fetch
    :: forall a t
    . String            -- name of the primitive stream
    -> HList t          -- tuple of attribute names, given by `Attribute`
    -> PrimitiveStream a (HList t)

Attribute names can be given by Attribute:

Attribute :: String -> t

Note that currently the type of the attributes are not statically checked.

Interface for outputs

Output is an existential type which wraps a framebuffer.

data Output where
  ScreenOut           :: FrameBuffer a b -> Output

renderFrame = ScreenOut