\chapter{Literature Review}\label{Background}
-This chapter will \rephrase{review the literature. It will also include some figures created by us from our test programs to aid with conceptual understanding of the literature.} A paper by paper summary of the literature is also available at: \\ \url{http://szmoore.net/ipdf/documents/LiteratureNotes.pdf}.
+The first part of this chapter will be devoted to documents themselves, including: the representation and displaying of graphics primitives, and how collections of these primitives are represented in document formats, focusing on widely used standards.
-\rephrase{TODO: If I want to link to the Paper by Paper summary it will need a bit of rewriting}.
+We will find that although there has been a great deal of research into the rendering, storing, editing, manipulation, and extension of document formats, modern standards are content to specify at best single precision IEEE-754 floating point arithmetic.
-\rephrase{TODO: Actually (re)write this entire chapter}.
+The research on arbitrary precision arithmetic applied to documents is rather sparse; however arbitrary precision arithmetic itself is a very active field of research. Therefore, remainder of this chapter will be devoted to considering fixed precision floating point numbers as specified by the IEEE-754 standard, possible limitations in precision, and alternative number representations for increased or arbitrary precision arithmetic.
-\rephrase{TODO: Un dot point ify}
+In Chapter \ref{Progress}, we will discuss our findings so far with regards to arbitrary precision arithmetic applied to document formats, and expand upon the goals outlined in Chapture \ref{Proposal}.
-\rephrase{TODO: Citations}
+\section{Raster and Vector Images}\label{Raster and Vector Images}
+\input{chapters/Background_Raster-vs-Vector}
-\rephrase{TODO: Make less terrible}
+\section{Rendering Vector Images}\label{Rasterising Vector Images}
+Hearn and Baker's textbook ``Computer Graphics''\cite{computergraphics2} gives a comprehensive overview of graphics from physical display technologies through fundamental drawing algorithms to popular graphics APIs. This section will examine algorithms for drawing two dimensional geometric primitives on raster displays as discussed in ``Computer Graphics'' and the relevant literature. This section is by no means a comprehensive survey of the literature but intends to provide some idea of the computations which are required to render a document.
-\rephrase{TODO: Reconsider sections (do I really want to make this go down to} \verb/\subsubsection/?)
+It is of some historical significance that vector display devices were popular during the 70s and 80s, and papers oriented towards drawing on these devices can be found\cite{brassel1979analgorithm}. Whilst curves can be drawn at high resolution on vector displays, a major disadvantage was shading\cite{lane1983analgorithm}; by the early 90s the vast majority of computer displays were raster based\cite{computergraphics2}.
-\rephrase{TODO: :-(}
+\subsection{Straight Lines}\label{Straight Lines}
+\input{chapters/Background_Lines}
+\subsection{Spline Curves and B{\'e}ziers}\label{Spline Curves}
+\input{chapters/Background_Spline}
+\subsection{Font Glyphs}\label{Font Rendering}
+\input{chapters/Background_Fonts}
-\section{Vector Graphics vs Raster Graphics}
+%\subsection{Shading}\label{Shading}
-\newlength\imageheight
-\newlength\imagewidth
-\settoheight\imageheight{\includegraphics{figures/fox-raster.png}}
-\settowidth\imagewidth{\includegraphics{figures/fox-raster.png}}
-%Height: \the\imageheight
-%Width: \the\imagewidth
+%\cite{brassel1979analgorithm}; %\cite{lane1983analgorithm}.
-\rephrase{TODO: Distinguish between Raster Formats and the Rasterisation of an image (which may or may not be in a raster format)}
+\subsection{Compositing}\label{Compositing}
+%So far we have discussed techniques for rendering vector graphics primitives in isolation, with no regard to the overall structure of a document which may contain many thousands of primitives. A straight forward approach would be to render all elements sequentially to the display, with the most recently drawn pixels overwriting lower elements. Such an approach is particularly inconvenient for anti-aliased images where colours must appear to smoothly blur between the edge of a primitive and any drawn underneath it.
-\subsubsection{Raster Graphics}
-\begin{itemize}
- \item Bitmap --- array of colour information for pixels
- \item Exact pixels in a similar format to how they would appear on a (modern) display device.
- \begin{itemize}
- \item Also similar to how they would be stored by a camera or scanner
- \item \rephrase{Is it misleading to say 2D array? Pixels are actually stored in a 1D array, but conceptually it's nicer to say 2D}
- \item \rephrase{For that matter, should it described as 3D (3rd dimension = colour)?}
- \end{itemize}
- \item Lowest level representation of a document
- \item Issues with scaling; values of extra pixels must be calculated
- \item Not convenient to edit; ill suited to text
-\end{itemize}
+Colour raster displays are based on an additive red-green-blue $(r,g,b)$ colour representation which matches the human eye's response to light\cite{computergraphics2}. In 1984, Porter and Duff introduced a fourth colour channel for rasterised images called the ``alpha'' channel, analogous to the transparency of a pixel\cite{porter1984compositing}. In compositing models, elements can be rendered seperately, with the four colour channels of successively drawn elements being combined according to one of several possible operations.
+In the ``painter's model'' as described by the SVG standard the ``over'' operation is used when rendering one primitive over another\cite{svg2011-1.1}.
+Given an existing pixel $P_1$ with colour values $(r_1, g_1, b_1, a_1)$ and a pixel $P_2$ with colours $(r_2, g_2, b_2, a_2)$ to be painted over $P_1$, the resultant pixel $P_T$ has colours given by:
+\begin{align}
+ a_T &= 1 - (1-a_1)(1-a_2) \\
+ r_T &= (1 - a_2)r_1 + r_2 \quad \text{(similar for $g_T$ and $b_T$)}
+\end{align}
+It should be apparent that alpha values of $1$ correspond to an opaque pixel; that is, when $a_2 = 1$ the resultant pixel $P_T$ is the same as $P_2$.
+When the final pixel is actually drawn on an rgb display, the $(r, g, b)$ components are $(r_T/a_T, g_T/a_T, b_T/a_T)$.
-\subsubsection{Vector Graphics}
-\begin{itemize}
- \item Stores relative position of primitives - scales better
- \item In particular, \emph{edges} of lines can be zoomed without becomming jagged; sometimes (somewhat misleadingly) described as ``infinitely sharp''
- \item Vector Graphics must be rasterised before being drawn on most display devices.
- \item Still can't scale forever due to use of fixed size floats
-\end{itemize}
+The PostScript and PDF standards, as well as the OpenGL API also use a painter's model for compositing. However, PostScript does not include an alpha channel, so $P_T = P_2$ always\cite{plrm}. Figure \ref{SVG} illustrates the painter's model for partially transparent shapes as they would appear in both the SVG and PDF models.
-\subsubsection{Resolution and Raster Graphics}
-\begin{itemize}
- \item DPI = dots (pixels) per inch differs per display device - a rastered image looks different on different display devices
- \item PostScript/PDF use 72 points per inch; this means a rasterised image will look the same in all pdf viewers regardless of the display.
- \item Tex uses 72.27 points per inch (?)
- \item The vector image was rastered at 96 points per inch
- \item Hence, have to scale by 72.27/96 = 0.7528125 to get the vector and rastered version to look exactly the same in the pdf
-\end{itemize}
+\subsection{Rasterisation on the CPU and GPU}
+Traditionally, vector images have been rasterized by the CPU before being sent to a specialised Graphics Processing Unit (GPU) for drawing\cite{computergraphics2}. Rasterisation of simple primitives such as lines and triangles have been supported directly by GPUs for some time through the OpenGL standard\cite{openglspec}. However complex shapes (including those based on B{\'e}zier curves such as font glyphs) must either be rasterised entirely by the CPU or decomposed into simpler primitives that the GPU itself can directly rasterise. There is a significant body of research devoted to improving the performance of rendering such primitives using the latter approach, mostly based around the OpenGL\cite{openglspec} API\cite{robart2009openvg, leymarie1992fast, frisken2000adaptively, green2007improved, loop2005resolution, loop2007rendering}. Recently Mark Kilgard of the NVIDIA Corporation described an extension to OpenGL for NVIDIA GPUs capable of drawing and shading vector paths\cite{kilgard2012gpu,kilgard300programming}. From this development it seems that rasterization of vector graphics may eventually become possible upon the GPU.
+It is not entirely clear how well supported the IEEE-754 standard for floating point computation is amongst GPUs\footnote{Informal technical articles are abundant on the internet --- Eg: Regarding the Dolphin Wii GPU Emulator: \url{https://dolphin-emu.org/blog} (accessed 2014-05-22)}. Although the OpenGL API does use IEEE-754 number representations, research by Hillesland and Lastra in 2004 suggested that many GPUs were not internally compliant with the standard\cite{hillesland2004paranoia}. %Arbitrary precision arithmetic, is provided by many software libraries for CPU based calculations
-\begin{figure}[H]
- \centering
-
- \includegraphics[scale=0.7528125]{figures/fox-vector.pdf}
- \includegraphics[scale=0.7528125]{figures/fox-raster.png}
- \includegraphics[scale=0.7528125, viewport=210 85 280 150,clip, width=0.45\textwidth]{figures/fox-vector.pdf}
- \includegraphics[scale=0.7528125, viewport=0 85 70 150,clip, width=0.45\textwidth]{figures/fox-raster.png}
- \caption{Scaling of vector and Raster Graphics}\label{vector-vs-raster}
-\end{figure}
-
-
-
-Figure \ref{vector-vs-raster} shows a vector image (left) which has been rasterised (right). At the original scale the two foxes should appear to be mirror images\footnote{If I've worked out the scaling to account for dpi differences between inkscape and latex/pdf correctly}. When the scale is increased, the edges of the vector image remain sharp, whilst the raster image begins to appear jagged. PDF viewers will typically use antialiasing to smooth the edges of a scaled bitmap, causing the image to appear blurred.\footnote{In the Atril Document Viewer 1.6.0 this image will only be antialiased at zoom levels $\leq 125\%$}.
-
-Various ways to end this section:
-\begin{enumerate}
- \item \rephrase{It should be obvious that documents containing text must use the vector graphics format, and so the remainder of this chapter will concentrate on the latter}.
- \item \rephrase{As can be seen in Figure \ref{fox}, if we were to decide to pursue ``infinite precision'' in raster graphics we would be shooting ourselves in both feet and then the face before we even started. The rest of this chapter will concentrate on vector graphics.}
- \item \rephrase{You can't have infinite precision in raster graphics by definition, therefore we no longer care about them in this report.}
- \item \rephrase{This report being in a vector format is a clue that we only care about vector formats}.
-\end{enumerate}
-
-\section{Primitives in Vector Graphics Formats (and how they are Rendered)}
-
-\subsection{Bezier Curves}
-\rephrase{I did an ipython notebook on this in February, but I forgot all of it}
-
-\subsection{Text}
-Text is just Bezier Curves
-
-\subsection{Shapes}
-Shapes are just bezier curves joined together.
-
-\subsection{Other Things}
-We don't really care about other things (ie: Colour gradients etc) in this report.
+ \pagebreak
+\section{Document Representations}\label{Document Representations}
-\section{Document Representations}
+The representation of information, particularly for scientific purposes, has changed dramatically over the last few decades. For example, Brassel's 1979 paper referenced earlier\cite{brassel1979analgorithm} has been produced on a mechanical type writer. Although the paper discusses an algorithm for shading on computer displays, the figures illustrating this algorithm have not been generated by a computer, but drawn by Brassel's assistant. In contrast, modern papers such as Barnes et. al's 2013 paper on embedding 3d images in PDF documents\cite{barnes2013embedding} can themselves be an interactive proof of concept.
-\rephrase{The file format can be either human readable\footnote{For some definition of human and some definition of readable} or binary\footnote{So, our viewer is basically a DOM style but stored in a binary format}. Can also be compressed or not. Here we are interested in how the document is interpreted or traversed in order to produce graphics output.}
+Haye's 2012 article ``Pixels or Perish'' discusses the recent history and current state of the art in documents for scientific publications\cite{hayes2012pixels}. Hayes argued that there are currently two different approaches to representing a document: As a sequence of static sheets of paper (Programmed Documents) or as a dynamic and interactive way to convey information, using the Document Object Model. We will now explore these two approaches and the extent to which they overlap.
+\subsection{Programmed Documents}
+\input{chapters/Background_Interpreted}
-\subsection{Interpreted Model}
+\pagebreak
+\subsection{Document Object Model}\label{Document Object Model}
+\input{chapters/Background_DOM}
-\rephrase{Did I just invent that terminology or did I read it in a paper? Is there actually existing terminology for this that sounds similar enough to ``Document Object Model'' for me to compare them side by side?}
+\subsection{The Portable Document Format}
-\begin{itemize}
- \item This model treats a document as the source code program which produces graphics
- \item Arose from the desire to produce printed documents using computers (which were still limited to text only displays).
- \item Typed by hand or (later) generated by a GUI program
- \item PostScript --- largely supersceded by PDF on the desktop but still used by printers\footnote{Desktop pdf viewers can still cope with PS, but I wonder if a smartphone pdf viewer would implement it?}
- \item \TeX --- Predates PostScript! {\LaTeX } is being used to create this very document and until now I didn't even have it here!
- \begin{itemize}
- \item I don't really want to go down the path of investigating the billion steps involved in getting \LaTeX into an actually viewable format
- \item There are interpreters (usually WYSIWYG editors) for \LaTeX though
- \item Maybe if \LaTeX were more popular there would be desktop viewers that converted \LaTeX directly into graphics
- \end{itemize}
- \item Potential for dynamic content, interactivity; dynamic PostScript, enhanced Postscript
+Adobe's Portable Document Format (PDF) is currently used almost universally for sharing documents; the ability to export or print to PDF can be found in most graphical document editors and even some plain text editors\cite{cheng2002finally}.
- \item Scientific Computing --- Mathematica, Matlab, IPython Notebook --- The document and the code that produces it are stored together
- \item Problems with security --- Turing complete, can be exploited easily
-\end{itemize}
+Hayes describes PDF as ``... essentially 'flattened' PostScript; it’s what’s left when you remove all the procedures and loops in a program, replacing them with sequences of simple drawing commands.''\cite{hayes2012pixels}. Consultation of the PDF 1.7 standard shows that this statement does not a give a complete picture --- despite being based on the Adobe PostScript model of a document as a series of ``pages'' to be printed by executing sequential instructions, from version 1.5 the PDF standard began to borrow some ideas from the Document Object Model. For example, interactive elements such as forms may be included as XHTML objects and styled using CSS. ``Actions'' are objects used to modify the data structure dynamically. In particular, it is possible to include Javascript Actions. Adobe defines the API for Javascript actions seperately to the PDF standard\cite{js_3d_pdf}. There is some evidence in the literature of attempts to exploit these features, with mixed success\cite{barnes2013embedding, hayes2012pixels}.
-\subsection{Crippled Interpreted Model}
+%\subsection{Scientific Computation Packages}
-\rephrase{I'm pretty sure I made that one up}
-\begin{itemize}
- \item PDF is PostScript but without the Turing Completeness
- \item Solves security issues, more efficient
-\end{itemize}
+\section{Precision required by Document Formats}
-\subsection{Document Object Model}
+We briefly summarise the requirements of the standards discussed so far in regards to the precision of mathematical operations.
-\begin{itemize}
- \item DOM = Tree of nodes; node may have attributes, children, data
- \item XML (SGML) is the standard language used to represent documents in the DOM
- \item XML is plain text
- \item SVG is a standard for a vector graphics language conforming to XML (ie: a DOM format)
- \item CSS style sheets represent more complicated styling on objects in the DOM
-\end{itemize}
+\subsection{PostScript}
+The PostScript reference describes a ``Real'' object for representing coordinates and values as follows: ``Real objects approximate mathematical real numbers within a much larger interval, but with limited precision; they are implemented as floating-point numbers''\cite{plrm}. There is no reference to the precision of mathematical operations, but the implementation limits \emph{suggest} a range of $\pm10^{38}$ ``approximate'' and the smallest values not rounded to zero are $\pm10^{-38}$ ``approximate''.
-\subsection{Blurring the Line --- Javascript}
+\subsection{PDF}
+PDF defines ``Real'' objects in a similar way to PostScript, but suggests a range of $\pm3.403\times10^{38}$ and smallest non-zero values of $\pm1.175\times10^{38}$\cite{pdfref17}. A note in the PDF 1.7 manual mentions that Acrobat 6 now uses IEEE-754 single precision floats, but ``previous versions used 32-bit fixed point numbers'' and ``... Acrobat 6 still converts floating-point numbers to fixed point for some components''.
-\begin{itemize}
- \item The document is expressed in DOM format using XML/HTML/SVG
- \item A Javascript program is run which can modify the DOM
- \item At a high level this may be simply changing attributes of elements dynamically
- \item For low level control there is canvas2D and even WebGL which gives direct access to OpenGL functions
- \item Javascript can be used to make a HTML/SVG interactive
- \begin{itemize}
- \item Overlooking the fact that the SVG standard already allows for interactive elements...
- \end{itemize}
- \item Javascript is now becoming used even in desktop environments and programs (Windows 8, GNOME 3, Cinnamon, Game Maker Studio) ({\bf shudder})
- \item There are also a range of papers about including Javascript in PDF ``Pixels or Perish'' being the only one we have actually read\cite{hayes2012pixels}
- \begin{itemize}
- \item I have no idea how this works; PDF is based on PostScript... it seems very circular to be using a programming language to modify a document that is modelled on being a (non turing complete) program
- \item This is yet more proof that people will converge towards solutions that ``work'' rather than those that are optimal or elegant
- \item I guess it's too much effort to make HTML look like PDF (or vice versa) so we could phase one out
- \end{itemize}
-\end{itemize}
+\subsection{\TeX and METAFONT}
-\subsection{Why do we still use static PDFs}
+In ``The METAFONT book'' Knuth appears to describe coordinates as fixed point numbers: ``The computer works internally with coordinates that are integer multiples of $\frac{1}{65536} \approx 0.00002$ of the width of a pixel''\cite{knuth1983metafont}. There is no mention of precision in ``The \TeX book''. In 2007 Beebe claimed that {\TeX} uses a $14.16$ fixed point encoding, and that this was due to the lack of standardised floating point arithmetic on computers at the time; a problem that the IEEE-754 was designed to solve\cite{beebe2007extending}. Beebe also suggests that \TeX and METAFONT could now be modified to use IEEE-754 arithmetic.
-Despite their limitations, we still use static, boring old PDFs. Particularly in scientific communication.
-\begin{itemize}
- \item They are portable; you can write an amazing document in Mathematica/Matlab but it
- \item Scientific journals would need to adapt to other formats and this is not worth the effort
- \item No network connection is required to view a PDF (although DRM might change this?)
- \item All rescources are stored in a single file; a website is stored accross many seperate files (call this a ``distributed'' document format?)
- \item You can create PDFs easily using desktop processing WYSIWYG editors; WYSIWYG editors for web based documents are worthless due to the more complex content
- \item Until Javascript becomes part of the PDF standard, including Javascript in PDF documents will not become widespread
- \item Once you complicate a PDF by adding Javascript, it becomes more complicated to create; it is simply easier to use a series of static figures than to embed a shader in your document. Even for people that know WebGL.
-\end{itemize}
+\subsection{SVG}
+The SVG standard specifies a minimum precision equivelant to that of ``single precision floats'' (presumably referring to IEEE-754) with a range of \verb/-3.4e+38F/ to \verb/+3.4e+38F/, and states ``It is recommended that higher precision floating point storage and computation be performed on operations such as
+coordinate system transformations to provide the best possible precision and to prevent round-off errors.''\cite{svg2011-1.1} An SVG Viewer may refer to itself as ``High Quality'' if it uses a minimum of ``double precision'' floats.
-\section{Precision in Modern Document Formats}
+\subsection{Javascript}
+%We include Javascript here due to its relation with the SVG, HTML5 and PDF standards.
+According to the EMCA-262 standard, ``The Number type has exactly 18437736874454810627 (that is, $2^64-^53+3$) values,
+representing the double-precision 64-bit format IEEE 754 values as specified in the IEEE Standard for Binary Floating-Point Arithmetic''\cite{ecma-262}.
+The Number type does differ slightly from IEEE-754 in that there is only a single valid representation of ``Not a Number'' (NaN). The EMCA-262 does not define an ``integer'' representation.
+
-\rephrase{All this is very interesting and provides important context, but it is not actually directly related to the problem of infinite precision which we are going to try and solve}.
+\section{Number Representations}\label{Number Representations}
+Consider a value of $7.25 = 2^2 + 2^1 + 2^0 + 2^{-2}$. In binary (base 2), this could be written as $111.01_2$ Such a value would require 5 binary digits (bits) of memory to represent exactly in computer hardware. Some values, for example $7.3$ can not be represented exactly in one base (decimal) but not another; in binary the sequence $111.010\text{...}_2$ will never terminate. A rational value such as $\frac{7}{3}$ could not be represented exactly in any base, but could be represented by the combination of a numerator $7 = 111_2$ and denominator $3 = 11_2$. Lastly, some values such as $e \approx 2.81\text{...}$ can only be expressed exactly using a symbolical system --- in this case as the result of an infinite summation --- $e = \displaystyle\sum_n=0^{\infty}\frac{1}{n!}$
-\begin{itemize}
- \item Implementations of PostScript and PDF must by definition restrict themselves to IEEE binary32 ``single precision''\footnote{The original IEEE-754 defined single, double and extended precisions; in the revision these were renamed to binary32, binary64 and binary128 to explicitly state the base and number of bits}
- floating point number representations in order to conform to the standards\cite{plrm, pdfref17}.
- \item Implementations of SVG are by definition required to use IEEE binary32 as a {\bf minimum}. ``High Quality'' SVG viewers are required to use at least IEEE binary64.\cite{svg2011-1.1}
- \item Numerical computation packages such as Mathematica and Maple use arbitrary precision floats
- \begin{itemize}
- \item Mathematica is not open source which is an issue when publishing scientific research (because people who do not fork out money for Mathematica cannot verify results)
- \item What about Maple? \cite{HFP} and \cite{fousse2007mpfr} both mention it being buggy.
- \item Octave and Matlab use fixed precision doubles
- \end{itemize}
-\end{itemize}
+Modern computer hardware typically supports integer and floating-point number representations and operations. Due to physical limitations, the size of these representations is limited; this is the fundamental source of both limits on range and precision in computer based calculations.
-The use of IEEE binary32 floats in the PostScript and PDF standards is not surprising if we consider that these documents are oriented towards representing static pages. They don't actually need higher precision to do this; 32 bits is more than sufficient for A4 paper.
+\subsection{Floating Point Definitions}
-\section{Representation of Numbers}
+Whilst a Fixed Point representation keeps the ``point'' at the same position in a string of bits, Floating point representations can be thought of as analogous to scientific notation; an ``exponent'' and fixed point value are encoded, with multiplication by the exponent moving the position of the point.
-Although this project has been motivated by a desire for more flexible document formats, the fundamental source of limited precision in vector document formats is the restriction to IEEE floating point numbers for representation of coordinates.
-
-Whilst David Gow will be focusing on structures \rephrase{and the use of multiple coordinate systems} to represent a document so as to avoid or reduce these limitations\cite{proposalGow}, the focus of our own research will be \rephrase{increased precision in the representation of real numbers so as to get away with using a single global coordinate system}.
-
-\subsection{The IEEE Standard}
-
-\subsection{Floating Point Number Representations}
+A floating point number $x$ is commonly represented by a tuple of values $(s, e, m)$ in base $B$ as\cite{HFP, ieee2008-754}:
\begin{align*}
x &= (-1)^{s} \times m \times B^{e}
\end{align*}
-$B = 2$, although IEEE also defines decimal representations for $B = 10$ --- these are useful in financial software\cite{ieee2008-754}.
-
-\rephrase{Aside: Are decimal representations for a document format eg: CAD also useful because you can then use metric coordinate systems?}
+Where $s$ is the sign and may be zero or one, $m$ is commonly called the ``mantissa'' and $e$ is the exponent. Whilst $e$ is an integer in some range $\pm e_max$, the mantissa $m$ is a fixed point value in the range $0 < m < B$.
-\subsubsection{Precision}
-The floats map an infinite set of real numbers onto a discrete set of representations.
+The choice of base $B = 2$ in the original IEEE-754 standard matches the nature of modern hardware. It has also been found that this base in general gives the smallest rounding errors\cite{HFP}. Early computers had in fact used a variety of representations including $B=3$ or even $B=7$\cite{goldman1991whatevery}, and the revised IEEE-754 standard specifies a decimal representation $B = 10$ intended for use in financial applications\cite{ieee754std2008}\footnote{Eg: The smallest valid unit of currency \$0.01 could not be represented exactly in base 2}. From now on we will restrict ourselves to considering base 2 floats.
-\rephrase{Figure: 8 bit ``minifloats'' (all 255 of them) clearly showing the ``precision vs range'' issue}
+The IEEE-754 encoding of $s$, $e$ and $m$ requires a fixed number of continuous bits dedicated to each value. Originally two encodings were defined: binary32 and binary64. $s$ is always encoded in a single leading bit, whilst (8,23) and (11,53) bits are used for the (exponent, mantissa) encodings respectively.
-The most a result can be rounded in conversion to a floating point number is the units in last place; $m_{N} \times B^{e}$.
+The encoding of $m$ in the IEEE-754 standard is not exactly equivelant to a fixed point value. By assuming an implicit leading bit (ie: restricting $1 \leq m < 2$) except for when $e = 0$, floating point values are gauranteed to have a unique representations; these representations are said to be ``normalised''. When $e = 0$ the leading bit is not implied; these representations are called ``denormals'' because multiple representations may map to the same real value. The idea of using an implicit bit appears to have been considered by Goldberg as early as 1967\cite{goldbern1967twentyseven}.
-\rephrase{Even though that paper that claims double is the best you will ever need because the error can be as much as the size of a bacterium relative to the distance to the moon}\cite{} \rephrase{there are many cases where increased number of bits will not save you}.\cite{HFP}
+Figure \ref{float.pdf}\footnote{In a digital PDF viewer we suggest increasing the zoom level --- the graphs were created from SVG images} shows the positive real numbers which can be represented exactly by an 8 bit floating point number encoded in the IEEE-754 format\footnote{Not quite; we are ignoring the IEEE-754 definitions of NaN and Infinity for simplicity}, and the distance between successive floating point numbers. We show two encodings using (1,2,5) and (1,3,4) bits to encode (sign, exponent, mantissa) respectively. For each distinct value of the exponent, the successive floating point representations lie on a straight line with constant slope. As the exponent increases, larger values are represented, but the distance between successive values increases; this can be seen on the right. The marked single point discontinuity at \verb/0x10/ and \verb/0x20/ occur when $e$ leaves the denormalised region and the encoding of $m$ changes. We have also plotted a fixed point representation for comparison; fixed point and integer representations appear as straight lines - the distance between points is always constant.
-\subsection{Limitations Imposed By CPU}
-
-CPU's are restricted in their representation of floating point numbers by the IEEE standard.
+The earlier example $7.25$ would be converted to a (1,3,4) floating point representation as follows:
+\begin{enumerate}
+ \item Determine the fixed point representation $7.25 = 111.01_2$
+ \item Determine the sign bit; in this case $s = 0$
+ \item Calculate the exponent by shifting the point $111.01_2 = 1.1101_2 \times 2^2 \implies e = 2 = 10_2$
+ \item Determine the exponent encoding; in IEEE-754 equal to the number of exponent bits is added so $e_{enc} = e+3 = 5 = 101_2$
+ \item Remove the implicit bit if the encoded exponent $\neq 0$; $1.1101_2 \to .1101_2$
+ \item Combine the three bit strings$0,101,1101$
+ \item The final encoding is $01011101 \equiv \text{0x5D}$
+\end{enumerate}
+This particular example can be encoded exactly; however as there are an infinite number of real values and only a finite number of floats, in general a value must be $7.26$ must be rounded or truncated at Step 3.
+\begin{figure}[H]
+ \centering
+\begin{minipage}[t]{0.45\textwidth}
+ \begin{figure}[H]
+ \centering
+ \includegraphics[width=1\textwidth]{figures/floats.pdf} \\
+ \end{figure}
+\end{minipage}
+\begin{minipage}[t]{0.45\textwidth}
+ \begin{figure}[H]
+ \centering
+ \includegraphics[width=1\textwidth]{figures/floats_diff.pdf} \\
+ \end{figure}
+\end{minipage}
+ \caption{8 bit float and fixed point representations a) As mapped to real values b) The distance between each representation}\label{floats.pdf}
+\end{figure}
-\subsection{Limitations Imposed By Graphics APIs and/or GPUs}
-Traditionally algorithms for drawing vector graphics are performed on the CPU; the image is rasterised and then sent to the GPU for rendering\cite{}. Recently there has been a great deal of literature relating to implementation of algorithms such as bezier curve rendering\cite{} or shading\cite{} on the GPU. As it seems the trend is to move towards GPU
-\begin{itemize}
- \item Internal representations are GPU dependent and may not match IEEE\cite{hillesland2004paranoia}
- \item OpenGL standards specify: binary16, binary32, binary64
- \item OpenVG aims to become a standard API for SVG viewers but the API only uses binary32 and hardware implementations may use less than this internally\cite{rice2008openvg}
- \item It seems that IEEE has not been entirely successful; although all modern CPUs and GPUs are able to read and write IEEE floating point types, many do not conform to the IEEE standard in how they represent floating point numbers internally.
-\end{itemize}
+\subsection{Precision and Rounding}\label{Precision and Rounding}
+Real values which cannot be represented exactly in a floating point representation must be rounded to the nearest floating point value. The results of a floating point operation will in general be such values and thus there is a rounding error possible in any floating point operation. Referring to Figure \ref{floats.pdf} it can be seen that the largest possible rounding error is half the distance between successive floats; this means that rounding errors increase as the value to be represented increases.
+Goldberg's assertively titled 1991 paper ``What Every Computer Scientist Needs to Know about Floating Point Arithmetic''\cite{goldberg1991whatevery} provides a comprehensive overview of issues in floating point arithmetic and relates these to requirements of the IEEE-754 1985 standard\cite{ieee754std1985}. More recently, after the release of the revised IEEE-754 standard in 2008\cite{ieee754std2008}, a textbook ``Handbook Of Floating Point Arithmetic'' has been published which provides a thourough review of literature relating to floating point arithmetic in both software and hardware\cite{HFP}.
-\rephrase{AND THUS WE FINALLY GET TO THE POINT}
+William Kahan, one of the architects of the IEEE-754 standard in 1984 and a contributor to its revision in 2010, has also published many articles on his website explaining the more obscure features of the IEEE-754 standard and calling out software which fails to conform to the standard\footnote{In addition to encodings and acceptable rounding behaviour, the standard also specifies ``exceptions'' --- mechanisms by which a program can detect and report an error such as division by zero}\cite{kahanweb, kahan1996ieee754}, as well as examples of the limitations of floating point computations\cite{kahan2007wrong}.
+In Figure \ref{calculatepi.pdf} we show the effect of accumulated rounding errors on the computation of $\pi$ through a numerical integration\footnote{This is not intended to be an example of a good way to calculate $\pi$} using 32 bit ``single precision'' floats and 64 bit ``double precision'' floats.
-\subsection{Examples of Precision Related Errors in Floating Point Arithmetic}
+\begin{figure}[H]
+ \centering
+ \includegraphics[width=0.6\textwidth]{figures/calculatepi.pdf}
+ \caption{Numerical calculation of $\pi$}\label{calculatepi.pdf}
+\end{figure}
-\subsection{Relate This to the Sorts of Maths Done By Document Formats}
+\subsection{Floating Point Operations}
-\subsection{Techniques for Arbitrary Precision Arithmetic}
+Floating point operations can in principle be performed using integer operations, but specialised Floating Point Units (FPUs) are an almost universal component of modern processors\cite{kelley1997acmos}. The improvement of FPUs remains highly active in several areas including: efficiency\cite{seidel2001onthe}; accuracy of operations\cite{dieter2007lowcost}; and even the adaptation of algorithms originally used in software, such as Kahan's Fast2Sum algorithm\cite{kadric2013accurate}.
-\begin{itemize}
- \item Fast2SUM for summation (and multiplication).
- \item Guard digits.
- \item Other techniques
- \item Hardware techniques that improve speed (which may be beneficial because you can get away with higher precision in hardware)
- \item Anything you can do in hardware you can do in software but it will be slower and have more segmentation faults
-\end{itemize}
+In 1971 Dekker formalised a series of algorithms including the Fast2Sum method for calculating the correction term due to accumulated rounding errors\cite{dekker1971afloating}. The exact result of $x + y$ may be expressed in terms of floating point operations with rounding as follows:
+\begin{align*}
+ z = \text{RN}(x + y) &\quad w = \text{RN}(z - x) \\
+ zz = \text{RN}(y - w) &\quad \implies x + y = zz
+\end{align*}
-\subsection{Alternate Number Representations}
+\subsection{Arbitrary Precision Floating Point Numbers}
-\rephrase{They exist\cite{HFP}}.
+Arbitrary precision floating point numbers are implemented in a variety of software libraries which will dynamically allocate extra bits for the exponent or mantissa as required. An example is the GNU MPFR library discussed by Fousse in 2007\cite{fouse2007mpfr}. Although many arbitrary precision libraries already existed, MPFR intends to be fully compliant with some of the more obscure IEEE-754 requirements such as rounding rules and exceptions.
+As we have seen, it is trivial to find real numbers that would require an infinite number of bits to represent exactly. Implementations of ``arbitrary'' precision must carefully determine at what point rounding should occur so as to balance performance with memory usage.
git.ucc.asn.au / ipdf / sam.git / blobdiff
UCC git Repository :: git.ucc.asn.au