The first half of this chapter will be devoted to documents themselves, including: the representation and displaying of graphics primitives\cite{computergraphics2}, and how collections of these primitives are represented in document formats, focusing on widely used standards\cite{plrm, pdfref17, svg2011-1.1}.
-We will find that although there has been a great deal of research into the rendering, storing, editing, manipulation, and extension of document formats, modern standards are content to specify at best single precision IEEE-754 floating point arithmetic.
+We will find that although there has been a great deal of research into the rendering, storing, editing, manipulation, and extension of document formats, modern standards are content to specify --- at best --- single precision IEEE-754 floating point arithmetic.
-The research on arbitrary precision arithmetic applied to documents is very sparse; however arbitrary precision arithmetic itself is a very active field of research. Therefore, the second half of this chapter will be devoted to considering fixed precision floating point numbers as specified by the IEEE-754 standard, possible limitations in precision, and alternative number representations for increased or arbitrary precision arithmetic.
+The research on arbitrary precision arithmetic applied to documents is rather sparse; however arbitrary precision arithmetic itself is a very active field of research. Therefore, the second half of this chapter will be devoted to considering fixed precision floating point numbers as specified by the IEEE-754 standard, possible limitations in precision, and alternative number representations for increased or arbitrary precision arithmetic.
In Chapter \ref{Progress}, we will discuss our findings so far with regards to arbitrary precision arithmetic applied to document formats, and expand upon the goals outlined in Chapture \ref{Proposal}.
\section{Rendering Vector Images}\label{Rasterising Vector Images}
-Hearn and Baker's textbook ``Computer Graphics''\cite{computergraphics2} gives a comprehensive overview of graphics from physical display technologies through fundamental drawing algorithms to popular graphics APIs. This section will examine algorithms for drawing two dimensional geometric primitives on raster displays as discussed in ``Computer Graphics'' and the relevant literature. Informal tutorials are abundant on the internet\cite{elias2000graphics}. This section is by no means a comprehensive survey of the literature but intends to provide some idea of the computations which are required to render a document.
+Hearn and Baker's textbook ``Computer Graphics''\cite{computergraphics2} gives a comprehensive overview of graphics from physical display technologies through fundamental drawing algorithms to popular graphics APIs. This section will examine algorithms for drawing two dimensional geometric primitives on raster displays as discussed in ``Computer Graphics'' and the relevant literature. This section is by no means a comprehensive survey of the literature but intends to provide some idea of the computations which are required to render a document.
It is of some historical significance that vector display devices were popular during the 70s and 80s, and papers oriented towards drawing on these devices can be found\cite{brassel1979analgorithm}. Whilst curves can be drawn at high resolution on vector displays, a major disadvantage was shading; by the early 90s the vast majority of computer displays were raster based\cite{computergraphics2}.
\subsection{Straight Lines}\label{Rasterising Straight Lines}
\input{chapters/Background_Lines}
-\subsection{Spline Curves}\label{Spline Curves}
+\subsection{Spline Curves and B{\'e}ziers}\label{Spline Curves}
+\input{chapters/Background_Spline}
-Splines are continuous curves formed from piecewise polynomial segments. A polynomial of $n$th degree is defined by $n$ constants $\{a_0, a_1, ... a_n\}$ and:
-\begin{align}
- y(x) &= \displaystyle\sum_{k=0}^n a_k x^k
-\end{align}
-
-Splines may be rasterised by sampling of $y(x)$ at a number of points $x_i$ and rendering straight lines between $(x_i, y_i)$ and $(x_{i+1}, y_{i+1})$ as discussed in Section \ref{Straight Lines}.
-
-There are many different ways to define a spline. One approach is to specify ``knots'' on the curve and choosing a fixed $n$ ($n = 3$ for ``cubic'' splines) solve for the cooefficients to generate polynomials passing through the points. Alternatively, special polynomials may be defined using ``control'' points which themselves are not part of the curve; these are convenient for graphical based editors. Bezier splines are the most straight forward way to define a curve in the standards considered in Section \ref{Document Representations}. A spline defined from two cubic beziers is shown in Figure \ref{spline.pdf}
-
-\begin{figure}[H]
-\centering
-\begin{minipage}[t]{0.3\textwidth}
-\begin{figure}[H]
- \centering
- \includegraphics[width=\textwidth]{figures/spline_labelled.pdf}
-\end{figure}
-\end{minipage}
-\begin{minipage}[t]{0.3\textwidth}
-\begin{minted}{xml}
-<!-- DOM element in SVG used to construct the spline -->
-<path d="M 0,300
- C 0,300 200,210 90,140
- -20,70 200,0 200,0"
- style="stroke:#000000; stroke-width:1px;
- fill:none;"/>
-\end{minted}
-\begin{minted}{postscript}
-% PostScript commands for a similar spline
-0 300 moveto
-0 300 200 210 90 140 curveto
--20 70 200 0 200 0 curveto stroke
-\end{minted}
-\end{minipage}
-\begin{minipage}[t]{0.3\textwidth}
-\begin{figure}[H]
- \centering
- \includegraphics[width=\textwidth]{figures/spline.pdf}
-\end{figure}
-\end{minipage}
- \caption{Constructing a Spline from two cubic beziers \\ (a) Showing the Control Points (b) Representations in SVG and PostScript (c) Rendered Spline}\label{spline.pdf}
-\end{figure}
-\subsubsection{Bezier Curves}
-\input{chapters/Background_Bezier}
-
-\subsection{Font Rendering}\label{Font Rendering}
+\subsection{Font Glyphs}\label{Font Rendering}
\input{chapters/Background_Fonts}
+%\subsection{Shading}\label{Shading}
-\subsection{Shading}
-
-Algorithms for shading on vector displays involved drawing equally spaced lines in the region with endpoints defined by the boundaries of the region\cite{brassel1979analgorithm}. Apart from being unrealistic, these techniques required a computationally expensive sorting of vertices\cite{lane1983analgorithm}.
+%\cite{brassel1979analgorithm}; %\cite{lane1983analgorithm}.
-On raster displays, shading is typically based upon Lane's algorithm of 1983\cite{lane1983analgorithm}. Lane's algorithm relies on the ability to ``subtract'' fill from a region. This algorithm is now implemented in the GPU \rephrase{stencil buffer-y and... stuff} \cite{kilgard2012gpu}
+\subsection{Compositing}\label{Compositing and the Painter's Model}
-\subsection{Compositing and the Painter's Model}\label{Compositing and the Painter's Model}
-
-So far we have discussed techniques for rendering vector graphics primitives in isolation, with no regard to the overall structure of a document which may contain many thousands of primitives. A straight forward approach would be to render all elements sequentially to the display, with the most recently drawn pixels overwriting lower elements. Such an approach is particularly inconvenient for anti-aliased images where colours must appear to smoothly blur between the edge of a primitive and any drawn underneath it.
+%So far we have discussed techniques for rendering vector graphics primitives in isolation, with no regard to the overall structure of a document which may contain many thousands of primitives. A straight forward approach would be to render all elements sequentially to the display, with the most recently drawn pixels overwriting lower elements. Such an approach is particularly inconvenient for anti-aliased images where colours must appear to smoothly blur between the edge of a primitive and any drawn underneath it.
Colour raster displays are based on an additive red-green-blue $(r,g,b)$ colour representation which matches the human eye's response to light\cite{computergraphics2}. In 1984, Porter and Duff introduced a fourth colour channel for rasterised images called the ``alpha'' channel, analogous to the transparency of a pixel\cite{porter1984compositing}. In compositing models, elements can be rendered seperately, with the four colour channels of successively drawn elements being combined according to one of several possible operations.
-In the ``painter's model'' as described by the SVG standard, Porter and Duff's ``over'' operation is used when rendering one primitive over another\cite{svg2011-1.1}.
+In the ``painter's model'' as described by the SVG standard the ``over'' operation is used when rendering one primitive over another\cite{svg2011-1.1}.
Given an existing pixel $P_1$ with colour values $(r_1, g_1, b_1, a_1)$ and a pixel $P_2$ with colours $(r_2, g_2, b_2, a_2)$ to be painted over $P_1$, the resultant pixel $P_T$ has colours given by:
\begin{align}
a_T &= 1 - (1-a_1)(1-a_2) \\
\subsection{Rasterisation on the CPU and GPU}
-Traditionally, vector graphics have been rasterized by the CPU before being sent to the GPU for drawing\cite{kilgard2012gpu}. Lots of people would like to change this \cite{worth2003xr, loop2007rendering, rice2008openvg, kilgard2012gpu, green2007improved}.
+Traditionally, vector images have been rasterized by the CPU before being sent to a specialised Graphics Processing Unit (GPU) for drawing\cite{computergraphics2}. Rasterisation of simple primitives such as lines and triangles have been supported directly by GPUs for some time through the OpenGL standard\cite{openglspec}. However complex shapes (including those based on B{\'e}zier curves such as font glyphs) must either be rasterised entirely by the CPU or decomposed into simpler primitives that the GPU itself can directly rasterise. There is a significant body of research devoted to improving the performance of rendering such primitives using the latter approach, mostly based around the OpenGL API\cite{robart2009openvg, leymarie1992fast, frisken2000adaptively, green2007improved, loop2005resolution, loop2007rendering}. Recently Mark Kilgard of the NVIDIA Corporation described an extension to OpenGL for NVIDIA GPUs capable of drawing and shading vector paths\cite{kilgard2012gpu,kilgard300programming}. From this development it seems that rasterization of vector graphics may eventually become possible upon the GPU.openglspec
-\rephrase{2. Here are the ways documents are structured ... we got here eventually}
+It is not entirely clear how well supported the IEEE-754 standard for floating point computation (which we will discuss in Section \ref{}) is amongst GPUs. Although the OpenGL API does use IEEE-754 number representations, research by Hillesland and Lastra in 2004 suggested that many GPUs were not internally compliant with the standard\cite{hillesland2004paranoia}. Arbitrary precision arithmetic, whilst provided by many libraries for CPU based calculations, is virtually unheard of in the context of GPU rendering.
+ \pagebreak
\section{Document Representations}\label{Document Representations}
-The representation of information, particularly for scientific purposes, has changed dramatically over the last few decades. For example, Brassel's 1979 paper referenced earlier has been produced on a mechanical type writer. Although the paper discusses an algorithm for shading on computer displays, the figures illustrating this algorithm have not been generated by a computer, but drawn by Brassel's assistant\cite{brassel1979analgorithm}. In contrast, modern papers such as Barnes et. al's recent paper on embedding 3d images in PDF documents\cite{barnes2013embeddding} can themselves be an interactive proof of concept.
-
-In this section we will consider various approaches and motivations to specifying the structure and appearance of a document, including: early interpreted formats (PostScript, \TeX, DVI), the Document Object Model popular in standards for web based documents (HTML, SVG), and Adobe's ubiquitous Portable Document Format (PDF). Some of these formats were discussed in a recent paper ``Pixels Or Perish'' by Hayes\cite{hayes2012pixelsor} who argues for greater interactivity in the PDF standard.
+The representation of information, particularly for scientific purposes, has changed dramatically over the last few decades. For example, Brassel's 1979 paper referenced earlier\cite{brassel1979analgorithm} has been produced on a mechanical type writer. Although the paper discusses an algorithm for shading on computer displays, the figures illustrating this algorithm have not been generated by a computer, but drawn by Brassel's assistant. In contrast, modern papers such as Barnes et. al's 2013 paper on embedding 3d images in PDF documents\cite{barnes2013embedding} can themselves be an interactive proof of concept.
-\subsection{Interpreted Document Formats}
-\input{chapters/Background_Interpreted}
+Haye's 2012 article ``Pixels or Perish'' discusses the recent history and current state of the art in documents for scientific publications\cite{hayes2012pixels}. Hayes argued that there are currently two different approaches to representing a document: As a sequence of static sheets of paper (Programmed Documents) or as a dynamic and interactive way to convey information, using the Document Object Model. We will now explore these two approaches and the extent to which they overlap.
-\begin{itemize}
- \item This model treats a document as the source code program which produces graphics
- \item Arose from the desire to produce printed documents using computers (which were still limited to text only displays).
- \item Typed by hand or (later) generated by a GUI program
- \item PostScript --- largely supersceded by PDF on the desktop but still used by printers\footnote{Desktop pdf viewers can still cope with PS, but I wonder if a smartphone pdf viewer would implement it?}
- \item \TeX --- Predates PostScript, similar idea
- \begin{itemize}
- \item Maybe if \LaTeX were more popular there would be desktop viewers that converted \LaTeX directly into graphics
- \end{itemize}
- \item Potential for dynamic content, interactivity; dynamic PostScript, enhanced Postscript
-
- \item Problems with security --- Turing complete, can be exploited easily
-\end{itemize}
+\subsection{Programmed Documents}
+\input{chapters/Background_Interpreted}
\pagebreak
\subsection{Document Object Model}\label{Document Object Model}
\subsection{The Portable Document Format}
-Adobe's Portable Document Format (PDF) is used almost universally for sharing documents; the ability to export or print to PDF can be found in most graphical document editors, even text editors.
+Adobe's Portable Document Format (PDF) is currently used almost universally for sharing documents; the ability to export or print to PDF can be found in most graphical document editors and even some plain text editors\cite{cheng2002finally}.
-Hayes describes PDF as ``... essentially 'flattened' PostScript; it’s what’s left when you remove all the procedures and loops in a program, replacing them with sequences of simple drawing commands.''\cite{hayes2012pixelsor}. Consultation of the PDF 1.7 standard shows that this statement does not a give a complete picture --- despite being based on the Adobe PostScript model of a document as a series of ``pages'' to be printed by executing sequential instructions, from version 1.5 the PDF standard began to borrow some ideas from the Document Object Model discussed in Section \ref{Document Object Model}. For example, interactive elements such as forms may be included as XHTML objects and styled using CSS. ``Actions'' are objects used to modify the data structure dynamically. In particular, it is possible to include Javascript Actions. Adobe defines the API for Javascript actions seperately to the PDF standard\cite{js_3d_pdf}. There is evidence in the literature of attempts to exploit these features, with mixed success\cite{barnes2013embedding, hayes2012pixelsor}.
+Hayes describes PDF as ``... essentially 'flattened' PostScript; it’s what’s left when you remove all the procedures and loops in a program, replacing them with sequences of simple drawing commands.''\cite{hayes2012pixels}. Consultation of the PDF 1.7 standard shows that this statement does not a give a complete picture --- despite being based on the Adobe PostScript model of a document as a series of ``pages'' to be printed by executing sequential instructions, from version 1.5 the PDF standard began to borrow some ideas from the Document Object Model. For example, interactive elements such as forms may be included as XHTML objects and styled using CSS. ``Actions'' are objects used to modify the data structure dynamically. In particular, it is possible to include Javascript Actions. Adobe defines the API for Javascript actions seperately to the PDF standard\cite{js_3d_pdf}. There is some evidence in the literature of attempts to exploit these features, with mixed success\cite{barnes2013embedding, hayes2012pixels}.
-To quote Adobe's PDF 1.7 reference manual, ``A PDF file should be thought of as a flattened representation of a data structure consisting of a collection of objects that can refer to each other in any arbitrary way''\cite{pdfref17}.
+%\subsection{Scientific Computation Packages}
+\section{Precision required by Document Formats}
-\subsection{Scientific Computation Packages}
+We briefly summarise the requirements of the standards discussed so far in regards to the precision of mathematical operations.
-The document and the code that produces it are one and the same.
+\subsection{PostScript}
+The PostScript reference describes a ``Real'' object for representing coordinates and values as follows: ``Real objects approximate mathematical real numbers within a much larger interval, but with limited precision; they are implemented as floating-point numbers''\cite{plrm}. There is no reference to the precision of mathematical operations, but the implementation limits \emph{suggest} a range of $\pm10^{38}$ ``approximate'' and the smallest values not rounded to zero are $\pm10^{-38}$ ``approximate''.
-\begin{itemize}
- \item Numerical computation packages such as Mathematica and Maple use arbitrary precision floats
- \begin{itemize}
- \item Mathematica is not open source which is an issue when publishing scientific research (because people who do not fork out money for Mathematica cannot verify results)
- \item What about Maple? \cite{HFP} and \cite{fousse2007mpfr} both mention it being buggy.
- \item Octave and Matlab use fixed precision doubles
- \end{itemize}
- \item IPython is pretty cool guys
-\end{itemize}
+\subsection{PDF}
+PDF defines ``Real'' objects in a similar way to PostScript, but suggests a range of $\pm3.403\times10^{38}$ and smallest non-zero values of $\pm1.175\times10^{38}$\cite{pdfref17}. A note in the PDF 1.7 manual mentions that Acrobat 6 now uses IEEE-754 single precision floats, but ``previous versions used 32-bit fixed point numbers'' and ``... Acrobat 6 still converts floating-point numbers to fixed point for some components''.
-\section{Precision in Modern Document Formats}
+\subsection{\TeX and METAFONT}
-We briefly summarise the requirements of the standards discussed so far in regards to the precision of mathematical operations:
-\begin{itemize}
- \item {\bf PostScript} predates the IEEE-754 standard and originally specified a floating point representation with ? bits of exponent and ? bits of mantissa. Version ? of the PostScript standard changed to specify IEEE-754 binary32 ``single precision'' floats.
- \item {\bf PDF} has also specified IEEE-754 binary32 since version ?. Importantly, the standard states that this is a \emph{maximum} precision; documents created with higher precision would not be viewable in Adobe Reader.
- \item {\bf SVG} specifies a minimum of IEEE-754 binary32 but recommends more bits be used internally
- \item {\bf Javascript} uses binary32 floats for all operations, and does not distinguish between integers and floats.
- \item {\bf Python} uses binary64 floats
- \item {\bf Matlab} uses binary64 floats
- \item {\bf Mathematica} uses some kind of terrifying symbolic / arbitrary float combination
- \item {\bf Maple} is similar but by many accounts horribly broken
-
-\end{itemize}
+In ``The METAFONT book'' Knuth appears to describe coordinates as fixed point numbers: ``The computer works internally with coordinates that are integer multiples of $\frac{1}{65536} \approx 0.00002$ of the width of a pixel''\cite{knuth1983metafont}. There is no mention of precision in ``The \TeX book''. In 2007 Beebe claimed this was due to the lack of standardised floating point arithmetic on computers at the time; a problem that the IEEE-754 was designed to solve\cite{beebe2007extending}. Beebe also suggests that \TeX and METAFONT could now be modified to use IEEE-754 arithmetic.
+
+\subsection{SVG}
+The SVG standard specifies a minimum precision equivelant to that of ``single precision floats'' (presumably referring to IEEE-754) with a range of \verb/-3.4e+38F/ to \verb/+3.4e+38F/, and states ``It is recommended that higher precision floating point storage and computation be performed on operations such as
+coordinate system transformations to provide the best possible precision and to prevent round-off errors.''\cite{svg2011-1.1} An SVG Viewer may refer to itself as ``High Quality'' if it uses a minimum of ``double precision'' floats.
-\rephrase{4. Here is IEEE-754 which is what these standards use}
+\subsection{Javascript}
+Although Javascript is not a stand alone document format, we include it here due to its relation with the SVG, HTML5 and PDF standards.
+
+
+According to the EMCA-262 standard, ``The Number type has exactly 18437736874454810627 (that is, $2^64-^53+3$) values,
+representing the double-precision 64-bit format IEEE 754 values as specified in the IEEE Standard for Binary Floating-Point Arithmetic''\cite{ecma-262}.
+The Number type does differ slightly from IEEE-754 in that there is only a single valid representation of ``Not a Number'' (NaN). The EMCA-262 does not define an ``integer'' representation.
+
\section{Real Number Representations}
\subsection{Limitations Imposed By Graphics APIs and/or GPUs}
-Traditionally algorithms for drawing vector graphics are performed on the CPU; the image is rasterised and then sent to the GPU for rendering\cite{}. Recently there has been a great deal of literature relating to implementation of algorithms such as bezier curve rendering\cite{} or shading\cite{} on the GPU. As it seems the trend is to move towards GPU
+Traditionally algorithms for drawing vector graphics are performed on the CPU; the image is rasterised and then sent to the GPU for rendering\cite{}. Recently there has been a great deal of literature relating to implementation of algorithms such as B{\'e}zier curve rendering\cite{} or shading\cite{} on the GPU. As it seems the trend is to move towards GPU
\rephrase{6. Here are ways GPU might not be IEEE-754 --- This goes *somewhere* in here but not sure yet}
+++ /dev/null
-Cubic and Quadratic Bezier Splines are used to define curved paths in the PostScript\cite{plrm}, PDF\cite{pdfref17} and SVG\cite{svg2011-1.1} standards which we will discuss in Section \ref{Document Representations}. Cubic Beziers are also used to define vector fonts for rendering text\cite{knuth1983metafont} (See Section \ref{Fonts}).
-
-A Bezier Curve of degree $n$ is defined by $n$ ``control points'' $\left\{P_0, ... P_n\right\}$.
-Points $P(t) = (x(t), y(t))$ along the curve are defined by:
-\begin{align}
- P(t) &= \displaystyle\sum_{j=0}^{n} B_j^n(t) P_j
-\end{align}
-Where $t \epsilon [0,1]$ is a control parameter. $P(0) = P_0$ and $P(1) = P_n$.
-
-A straightforward algorithm for rendering Bezier's is to simply sample $P(t)$ for suffiently many values of $t$ and connect the resulting points with straight lines using Bresenham or Wu's algorithm (See Section \ref{Straight Lines}). Whilst the performance of this algorithm is linear, De Casteljau derived a more efficient means of sub dividing beziers into line segments.
-
-Recently, Goldman presented an argument that Bezier's could be considered as fractal in nature, because the De Casteljau algorithm could be modified to be expressed as an iterated function system\cite{goldman_thefractal}.
-
-
The World Wide Web Consortium (W3C) is an organisation devoted to the development of standards for structuring and rendering web pages based on industry needs. The DOM is used in and described by several W3C recommendations including XML\cite{xml2008-1.0}, HTML\cite{html2014-draft} and SVG\cite{svg2011-1.1}. XML is a general language which is intended for representing any tree-like structure using the DOM, whilst HTML and SVG are specifically intended for representing visual information to humans. These languages make use of Cascading Style Sheets (CSS)\cite{css2011-level2} for specifying the appearance of elements.
-Version 5 of the Hypertext Markup Language (HTML5) is currently a candidate recommendation which aims to standardise the state of the art in technologies relating to web based documents. In HTML5 it is possible to achieve almost any level of control over both the structure and rendering of a document desirable. In particular, the interpreted language Javascript included in the HTML5 standard can be used to dynamically alter the document as it is viewed in response to user input or other events such as communication with a remote server.
-
-The Scalable Vector Graphics (SVG) recommendation defines a language for representing vector images using the DOM. This is intended not only for stand alone images, but also for inclusion within HTML documents. In the SVG standard, each graphics primitive is an element in the DOM, whilst attributes of the element give information about how the primitive is to be drawn, such as path coordinates, line thickness, mitre styles and fill colours. Figure \ref{SVG} shows an example of an SVG image as rendered (left) and represented as text. The textual representation is syntactically a subset of XML and is similar to HTML.\footnote{The exact details of classification of these languages (such as why HTML cannot be defined as a subset of XML) are beyond the scope of this report.} Here we have used \verb/<rect>/ elements to position rectangles and \verb/<path>/ elements to define a straight line and a filled region bounded by a cubic bezier spline; note that the points and type of curves are defined as a data attribute.
+Version 5 of the Hypertext Markup Language (HTML5) is currently a candidate recommendation which aims to standardise the state of the art in technologies relating to web based documents. In HTML5 it is possible to achieve almost any level of control over both the structure and rendering of a document desirable. In particular, the language Javascript (based upon ECMAScript \cite{ecma-262}) can be used to dynamically alter a HTML5 document in response to user input or other events, including communication with HTTP servers.
+The Scalable Vector Graphics (SVG) recommendation defines a language for representing vector images using the DOM. This is intended not only for stand alone images, but also for inclusion within HTML documents. In the SVG standard, each graphics primitive is an element in the DOM, whilst attributes of the element give information about how the primitive is to be drawn, such as path coordinates, line thickness, mitre styles and fill colours. Figure \ref{SVG} shows an example of an SVG image as rendered (left) and represented as text. The textual representation is syntactically a subset of XML and is similar to HTML.\footnote{The details of distinctions between these languages are beyond the scope of this report.} Here we have used \verb/<rect>/ elements to position rectangles and \verb/<path>/ elements to define a straight line and a filled region bounded by a cubic bezier spline; note that the points and type of curves are defined as a data attribute.
\subsubsection{Javascript and the DOM}
-The W3C has produced a primer describing the use of HTML5 Javascript to produce interactive SVG's\cite{w3c2010svghtmlprimer}, and the HTML5 and SVG standards themselves include several examples discussing the use of Javascript to manipulate the DOM.
-
-In Javascript, an element in the DOM can be selected by its type, class, name, or unique identifier, each of which may be specified as an attribute in the original DOM. Once an element is selected Javascript can be used to modify its attributes, add children below it in the DOM, or remove it from the DOM entirely.
+Using Javascript, an element in the DOM can be selected by its type, class, name, or unique identifier, each of which may be specified as an attribute in the original DOM. Once an element is selected Javascript can be used to modify its attributes, add children below it in the DOM, or remove it from the DOM entirely.
-For example, the following Javascript acting on the DOM described in Figure \ref{SVG} would change the fill colour of the curved shape from red \verb/#ff0000/ to black \verb/#000000/:
+For example, the following Javascript acting on the DOM described in Figure \ref{SVG} will change the fill colour of the curved region.
\begin{minted}{javascript}
var node = document.getElementById("curvedshape"); // Find the node by its unique id
-node.style.fill = "#000000"; // Change the ``style'' attribute and set the CSS fill colour
+node.style.fill = "#000000"; // Change the ``style'' attribute and set the CSS fill colour
\end{minted}
-To illustrate the power of this technique we have produced our own example to generate an SVG interactively using HTML. The example generates successive iterations of a particular type of fractal curve first described by Koch\cite{koch1904surune} in 1904 and a popular example in modern literature \cite{goldman_fractal}. Unfortunately as it is currently possible to directly include W3C HTML in a PDF, we are only able to provide some examples of the output as static images in Figure \ref{koch}.
+To illustrate the power of this technique we have produced an example to generate an SVG interactively using HTML. The example generates successive iterations of a particular type of fractal curve first described by Koch\cite{koch1904surune} in 1904 and a popular example in modern literature \cite{goldman_fractal}. Unfortunately as it is currently possible to directly include W3C HTML in a PDF, we are only able to provide some examples of the output as static images in Figure \ref{koch}. The W3C has produced a primer describing the use of HTML5 and Javascript to produce interactive SVG's\cite{w3c2010svghtmlprimer}, and the HTML5 and SVG standards themselves include several examples.
In HTML5, Javascript is not restricted to merely manipulating the DOM to alter the appearance of a document. The \verb/<canvas>/ tag and associated API provide a means to directly set the values of pixels on a display. This sort of low level API is inteded for performance intensive graphical applications such as web based games\footnote{For an example by the author including both the canvas2d and experimental WebGL APIs see \url{http://rabbitgame.net}}. As Hayes points out, there is some similarity between the \verb/<canvas>/ API and the PostScript interpreted approach to drawing\cite{hayes2012pixelsor}.
-A typeface or font refers to a set of images used to represent text on a graphical display\footnote{This terminology dates back to printing press technology}.
-In 1983, Donald Knuth published ``The METAFONT Book'' which described a vector approach to specifying fonts and a program for creating these fonts\footnote{knuth1983metafont}. Previously, only rasterised font images were popular; as can be seen from the zooming in Figure \ref{vector-vs-raster-scaled} this is problematic given the prevelance of textual information at different scales and on different resolution displays.
-
-Knuth used Bezier Cubic Splines as discussed in Section \ref{} to define ``pleasing'' curves in METAFONT, and this approach is still used in modern vector fonts. Since the paths used to render an individual character (often called ``glyphs'') are used far more commonly than general curves, document formats do not require such curves to be specified in situ, but allow for a choice between a number of internal fonts or externally specified fonts. In the case of Knuth's typesetting language \TeX, fonts were intended to be created using METAFONT\cite{knuth}.
-
-Figure \ref{zglyph} shows a $\mathscr{Z}$ (Z) in Ralph Smith's Formal Script font\footnote{\url{http://www.tug.dk/FontCatalogue/rfsf/}}. On the right, a screenshot taken in the Inkscape vector graphics editor shows the start/end points for each Cubic Bezier in the Spline defining the glyph outline\footnote{Inkscape activates the two adjacent control points after selecting one of the bezier start/end points; these are the two circles in Figure \ref{zglyph}}.
-
\begin{figure}[H]
\begin{minipage}[t]{0.5\textwidth}
\begin{figure}[H]
\centering
- \includegraphics[width=1\textwidth]{figures/z.pdf}
+ \includegraphics[width=0.7\textwidth]{figures/z.pdf}
\end{figure}
\end{minipage}
\begin{minipage}[t]{0.5\textwidth}
\begin{figure}[H]
\centering
- \includegraphics[width=1\textwidth]{figures/z.png}
+ \includegraphics[width=0.7\textwidth]{figures/z.png}
\end{figure}
\end{minipage}
- \caption{A glyph for the letter Z and a screenshot showing bezier start/end points as squares and diamonds}\label{zglyph}
+ \caption{a) Vector glyph for the letter Z b) Screenshot showing B{\'e}zier control points in Inkscape}
\end{figure}
+
+A the term ``font'' refers to a set of images used to represent text on a graphical display. In 1983, Donald Knuth published ``The METAFONT Book'' which described a vector approach to specifying fonts and a program for creating these fonts\cite{knuth1983metafont}. Previously, only rasterised font images (glyphs) were popular; as can be seen from the zooming in Figure \ref{vector-vs-raster-scaled} this can be problematic given the prevelance of textual information at different scales and on different resolution displays.
+
+Knuth used B{\'e}zier Cubic Splines to define ``pleasing'' curves in METAFONT, and this approach is still used in modern vector fonts. Since the paths used to render an individual glyph are used far more commonly than general curves, document formats do not require such curves to be specified in situ, but allow for a choice between a number of internal fonts or externally specified fonts. In the case of Knuth's typesetting language \TeX, fonts were intended to be created using METAFONT\cite{knuth}. Figure \ref{zglyph} shows a $\mathscr{Z}$ (Z) in Ralph Smith's Formal Script font as produced by \LaTeX with the start and end points of each Cubic B{\'e}zier identified.
+
+
-
\subsubsection{PostScript}
Adobe's PostScript Language Reference Manual defines a turing complete language for producing graphics output on an abstract ``output device''\cite{plrm}. A PostScript document is treated as a procedural program; an interpreter executes instructions in the order they are written by the programmer. Each symbol is pushed onto a stack as it is read. Special symbols called ``operators'' can act upon this stack and/or the output device. An internal ``graphics state'' stack can be constructed to store styling information (such as colour, line thickness, the current cursor position). It is possible for the language to define new operators. Figure \ref{PS} shows a vector image and one possible way to express this image in PostScript. PostScript was and is still widely used in printing of documents onto paper; many printers execute postscript directly, and newer formats including PDFs must still be converted into PostScript by printer drivers\cite{pdfref17, cheng2002portable}.
There are some limitations in PostScript's model. As mentioned in Section\ref{Compositing}, since PostScript predates Porter and Duff Compositing, there is no concept of transparency. In fact, using tools to convert between the SVG image in Figure \ref{SVG} and PostScript will simply rasterise the image and embed the rastered image in PostScript\footnote{For Figure \ref{SVG} converted using the Inkscape SVG editor: \url{http://szmoore.net/ipdf/figures/shape-svg-converted-to.ps}}
-Another limitation of PostScript is that the model of a document as a static page, convenient for printers which literally produce static pages, is unable to include interactive or dynamic elements. Dynamic PostScript attempted to fix this problem, but ``never caught on''\cite{hayes2012pixelsor}.
+Another limitation of PostScript is that the model of a document as a static page, convenient for printers which literally produce static pages, is unable to include interactive or dynamic elements. Dynamic PostScript attempted to fix this problem, but ``never caught on''\cite{hayes2012pixels}.
\begin{figure}[H]
\begin{minipage}[t]{0.65\textwidth}
\caption{Vector image and a possible PostScript representation}\label{PS}
\end{figure}
-\subsubsection{\TeX and Metafont}
+\subsubsection{{\TeX}, METAFONT and {\LaTeX}}
-\TeX is a typesetting language invented by Donald Knuth in 1983.
+Knuth's ``The {\TeX}book''\cite{knuth1984texbook} and ``The METAFONT book''\cite{knuth1983metafont} define two complementary programming languages for typesetting documents. Wheras PostScript may be considered an interpreted language, in that it can be produced in a human readable form which is also readable by an interpreter, {\TeX} is a compiled language; a program parses human readable {\TeX} to produce a machine readable format DVI (``DeVice Independent''). A DVI interpreter might be thought of as a virtual ``Display Processor'' for drawing vector graphics directly (as defined in the earlier editions of ``Computer Graphics''\cite{computergraphics2}).
+DVI itself is not a widely used format for sharing documents. However, an system based upon {\TeX} called {\LaTeX} which includes libraries for advanced typesetting and programs that ultimately produce PDF output is particularly popular for producing technical reports and papers\footnote{The site \url{http://tex.stackexchange.com} (accessed 2014-05-22) is devoted to {\TeX} and {\LaTeX}} --- this report itself has been produced using the CTAN {\LaTeX} packages\footnote{The complete {\TeX} source code to produce this document can be found at \url{http://szmoore.net/ipdf/sam/}}.
In Figure \ref{rasterising-line} a) and b) we illustrate the rasterisation of a line width a single pixel width. The path followed by Bresenham's algorithm is shown. It can be seen that the pixels which are more than half filled by the line are set by the algorithm. This causes a jagged effect called aliasing which is particularly noticable on low resolution displays. From a signal processing point of view this can be understood as due to the sampling of a continuous signal on a discrete grid\cite{wu1991anefficient}.
-Figure \ref{rasterising-line} c) shows an (idealised) antialiased rendering of the line. The pixel intensity has been set to the average of the line and background colours over that pixel. Such an ideal implementation would be impractically computationally expensive on real devices\cite{elias2000graphics}. In 1991 Wu introduced an algorithm for drawing anti-aliased lines which, while equivelant in results to existing algorithms by Fujimoto and Iwata, set the state of the art in performance\cite{wu1991anefficient}\footnote{Techniques for anti-aliasing primitives other than straight lines, including the anti-aliasing of images in a raster format shown at different scales (such as Figure \ref{}) are discussed in some detail in Chapter 4 of ``Computer Graphics'' \cite{computergraphics2}}.
+Figure \ref{rasterising-line} c) shows an (idealised) antialiased rendering of the line. The pixel intensity has been set to the average of the line and background colours over that pixel. Such an ideal implementation would be impractically computationally expensive on real devices\cite{elias2000graphics}. In 1991 Wu introduced an algorithm for drawing anti-aliased lines which, while equivelant in results to existing algorithms by Fujimoto and Iwata, set the state of the art in performance\cite{wu1991anefficient}\footnote{Techniques for anti-aliasing primitives other than straight lines are discussed in some detail in Chapter 4 of ``Computer Graphics'' \cite{computergraphics2}}.
.
At a fundamental level everything that is seen on a display device is represented as either a vector or raster image. These images can be stored as stand alone documents or embedded within a more complex document format capable of containing many other types of information.
-A raster image's structure closely matches it's representation as shown on modern display hardware; the image is represented as a grid of filled square ``pixels''. Each pixel is considered to be a filled square of the same size and contains information describing its colour. This representation is simple and also well suited to storing images as produced by cameras and scanners.
+A raster image's structure closely matches it's representation as shown on modern display hardware; the image is represented as a grid of filled square ``pixels''. Each pixel is considered to be a filled square of the same size and contains information describing its colour. This representation is simple and also well suited to storing images as produced by cameras and scanners. The drawback of raster images is that by their very nature there can only be one level of detail.
-The drawback of raster images is that by their very nature there can only be one level of detail. Figures \ref{vector-vs-raster} and \ref{vector-vs-raster-scaled} attempt to illustrate this by comparing raster images to vector images in a similar way to Worth and Packard\cite{worth2003xr}.
+A vector image contains information about the positioning and shading of geometric shapes. To display this image on modern display hardware, coordinates are transformed according to the view and then the image is converted into a raster like representation. Whilst the raster image merely appears to contain edges, the vector image actually contains information about these edges, meaning they can be displayed ``infinitely sharply'' at any level of detail\cite{citationneeded} --- or they could be if the coordinates are stored with enough precision (see Section \ref{}).
-The right side of Figure \ref{vector-vs-raster} is a raster image which should be recognisable as an animal defined by fairly sharp edges. Figure \ref{vector-vs-raster-scaled} shows how these edges appear jagged when scaled. There is no information in the original image as to what should be displayed at a larger size, so each square shaped pixel is simply increased in size. A blurring effect will probably be visible in most PDF viewers; the software has attempted to make the ``edge'' appear more realistic using a technique called ``antialiasing''.
-%(See Section \ref{Straight Lines}).\footnote{The exact appearance of the images at different zoom levels will depend greatly on the PDF viewer or printer used to display this report. On the author's display using the Atril (1.6.0) Document viewer, the top images appear to be pixel perfect mirror images at a 100\% scale. In the bottom raster image, antialiasing is not applied at zoom levels above $125\%$ and the effect of scaling is quite noticeable.}
+Figures \ref{vector-vs-raster} and \ref{vector-vs-raster-scaled} attempt to illustrate the advantage of vector formats by comparing raster and vector images in a similar way to Worth and Packard\cite{worth2003xr}. The right side of Figure \ref{vector-vs-raster} is a raster image which should be recognisable as an animal defined by fairly sharp edges. Figure \ref{vector-vs-raster-scaled} shows how these edges appear jagged when scaled. There is no information in the original image as to what should be displayed at a larger size, so each square shaped pixel is simply increased in size. A blurring effect will probably be visible in most PDF viewers; the software has attempted to make the ``edge'' appear more realistic using a technique called ``antialiasing''.
-%\footnote{\noindent This behaviour may be configured in some PDF viewers (Adobe Reader) whilst others (Evince, Atril, Okular) will choose whether or not to bother with antialiasing based on the zoom level. For best results experiment with changing the zoom level in your PDF viewer.\footnotemark}\footnotetext{On the author's hardware, the animals in the vector and raster images should appear mirrored pixel for pixel; but they may vary slightly on other PDF viewers or display devices.}
-
-In contrast, the left sides of Figures \ref{vector-vs-raster} and \ref{vector-vs-raster-scaled} are a vector image. A vector image contains information about the positioning and shading of geometric shapes. To display this image on modern display hardware, coordinates are transformed according to the view and then the image is converted into a raster like representation. Whilst the raster image merely appears to contain edges, the vector image actually contains information about these edges, meaning they can be displayed ``infinitely sharply'' at any level of detail\cite{citationneeded} --- or they could be if the coordinates are stored with enough precision (see Section \ref{}). Vector images are well suited to high quality digital art\footnote{Figure \ref{vector-vs-raster} is not to be taken as an example of this.} and text.
+The left sides of Figures \ref{vector-vs-raster} and \ref{vector-vs-raster-scaled} are a vector image. When scaled, the edges maintain a smooth appearance which is limited by the resolution of the display rather than the image itself. Vector images are well suited to high quality digital art\footnote{Figure \ref{vector-vs-raster} is not to be taken as an example of this.} and text.
\newlength\imageheight
--- /dev/null
+
+Splines are continuous curves formed from piecewise polynomial segments. A polynomial of $n$th degree is defined by $n$ constants $\{a_0, a_1, ... a_n\}$ and:
+\begin{align}
+ y(x) &= \displaystyle\sum_{k=0}^n a_k x^k
+\end{align}
+
+ \begin{comment}
+Splines may be rasterised by sampling of $y(x)$ at a number of points $x_i$ and drawing straight lines between $(x_i, y_i)$ and $(x_{i+1}, y_{i+1})$ as discussed in Section \ref{Straight Lines}.
+
+There are many different ways to define a spline.One approach is to specify ``knots'' on the curve and choosing a fixed $n$ ($n = 3$ for ``cubic'' splines) solve for the cooefficients to generate polynomials passing through the points. Alternatively, special polynomials may be defined using ``control'' points which themselves are not part of the curve; these are convenient for graphical based editors.\end{co B{\'e}zier splines are the most straight forward way to define a curve in the standards considered in Section \ref{Document Representations}. A spline defined from two cubic B{\'e}ziers is shown in Figure \ref{spline.pdf}
+\end{comment}
+
+Cubic and Quadratic B{\'e}zier Splines are used to define curved paths in the PostScript\cite{plrm}, PDF\cite{pdfref17} and SVG\cite{svg2011-1.1} standards which we will discuss in Section \ref{Document Representations}. Cubic B{\'e}ziers are also used to define vector fonts for rendering text in these standards and the \TeX typesetting language \cite{knuth1983metafont, knuth1984texbook}. The usefulness of B{\'e}zier curves was realised by Pierre B{\'e}zier who used them in the 1960s for the computer aided design of automobile bodies\cite{bezier1986apersonal}.
+
+A B{\'e}zier Curve of degree $n$ is defined by $n$ ``control points'' $\left\{P_0, ... P_n\right\}$.
+Points $P(t) = (x(t), y(t))$ along the curve are defined by:
+\begin{align}
+ P(t) &= \displaystyle\sum_{j=0}^{n} B_j^n(t) P_j
+\end{align}
+Where $t \epsilon [0,1]$ is a control parameter. The polynomials $B_j^n(t)$ are Bernstein Basis Polynomials which are defined as:
+\begin{align}
+ B_j^n(t) &= \left(^n_j\right) t^j\left(1-t\right)^{n-j} \quad \quad j=0,1,...,n \\
+ \text{Where } \left(^n_j\right) &= \frac{n!}{n!(n-j)!} \quad \text{ (The Binomial Coefficients)}
+\end{align}
+From these definitions it should be apparent that in all cases, $P(0) = P_0$ and $P(1) = P_n$. An $n = 1$ B{\'e}zier Curve is a straight line.
+
+Algorithms for rendering B{\'e}zier's may simply sample $P(t)$ for suffiently many values of $t$ --- enough so that the spacing between successive points is always less than one pixel distance. Alternately, a smaller number of points may be sampled with the resulting points connected by straight lines using one of the algorithms discussed in Section \ref{Straight Lines}.
+
+De Casteljau's algorithm of 1959 is often used for approximating B{\'e}ziers\cite{computergraphics2, knuth1983metafont}. This algorithm subdivides the original $n$ control points $\left\{P_0, ... P_n\right\}$ into $2n$ points $\left\{Q_0, ... Q_n\right\}$ and $\left\{R_0, ... R_n\right\}$; when iterated, the produced points will converge to $P(t)$. As a tensor equation this subdivision can be expressed as:
+\begin{align}
+ Q_i = \left(\frac{\left(^n_j\right)}{2^j}\right) P_i &\text{ and } R_i = \left(\frac{\left(^{n-j}_{n-k}\right)}{2^{n-j}}\right) P_i
+\end{align}
+
+
+In much of the literature it is taken as trivial that it is only necessary to specify the control points of a B{\'e}zier in order to be able to render it at any level of detail\cite{knuth1983metafont, computergraphics2}. Recently, Goldman presented an argument that B{\'e}zier's could be considered as fractal in nature, because the De Casteljau algorithm may be modified to be expressed the polynomial $P(t)$ as the result of iterated function system\cite{goldman_thefractal}. If this argument is correct, any primitive that can be described soley in terms of B{\'e}zier Curves may also be considered as fractal in nature. Ideally all these primitives may be rendered at any level of detail or ``zoom'' desired; however, computation of the pixel locations of the curve will be subject to the precision limits of the numerical representation which is used; we discuss these issues in Section \ref{}.
+
+
+\begin{figure}[H]
+\centering
+\begin{minipage}[t]{0.3\textwidth}
+\begin{figure}[H]
+ \centering
+ \includegraphics[width=0.7\textwidth]{figures/spline_labelled.pdf}
+\end{figure}
+\end{minipage}
+\begin{minipage}[t]{0.3\textwidth}
+\begin{minted}{xml}
+<!-- DOM element in SVG used to construct the spline -->
+<path d="M 0,300
+ C 0,300 200,210 90,140
+ -20,70 200,0 200,0"
+ style="stroke:#000000; stroke-width:1px;
+ fill:none;"/>
+\end{minted}
+\begin{minted}{postscript}
+% PostScript commands for a similar spline
+0 300 moveto
+0 300 200 210 90 140 curveto
+-20 70 200 0 200 0 curveto stroke
+\end{minted}
+\end{minipage}
+\begin{minipage}[t]{0.3\textwidth}
+\begin{figure}[H]
+ \centering
+ \includegraphics[width=0.7\textwidth]{figures/spline.pdf}
+\end{figure}
+\end{minipage}
+ \caption{Constructing a Spline from two cubic B{\'e}ziers \\ (a) Showing the Control Points (b) Representations in SVG and PostScript (c) Rendered Spline}\label{spline.pdf}
+\end{figure}
\chapter{Proposal}\label{Proposal}
-\rephrase{Most of this chapter is copy pasted from the project proposal} \\
- \url{http://szmoore.net/ipdf/documents/ProjectProposalSam.pdf}
-
\section{Aim}
-In this project, we will explore the state of the art of current document formats including PDF, PostScript, SVG, HTML, and the limitations of each in terms of precision.
-We will consider designs for a document format allowing graphics primitives at an arbitrary level of zoom with no loss of detail. A viewer and editor will be implemented as a proof of concept; we adopt a low level, ground up approach to designing this viewer so as to not become restricted by any single existing document format.
+In this project, we will explore the state of the art of current document formats including PDF, PostScript, SVG, HTML, and the limitations of each with regards to precision.
+
+We will consider designs for a document format allowing graphics primitives at an arbitrary level of zoom with no loss of detail. A viewer and editor will be implemented as a proof of concept; we adopt a low level, ground up approach to designing this viewer so as to not become restricted by any single existing document format. Although it is possible to produce three dimensional graphics using some of the technologies we will explore, we will focus on two dimensional graphics.
There are many possible applications for documents in which precision is unlimited. Several areas of use include: visualisation of extremely large or infinite data sets; visualisation of high precision numerical computations; digital artwork; computer aided design; and maps.
\subsection{Clarification of Terms}
It may be necessary to clarify what we mean by the terms ``arbitrary precision'' and ``document formats''. Regarding the latter, we consider a document format to be any representation of visual information which is capable of being stored indefinitely. Regarding the former, we do not propose to be able to contain an infinite amount of information within such a document. The goal is to be able to render a primitive at the same level of detail it is specified by a document format, regardless of how precise this level is. For example, the precision of coordinates of primitives drawn in a graphical document editor will always be limited by the resolution of the display on which they are drawn, but not by the viewer.
+
\section{Methods}
\item {\bf Arbitrary Precision real valued numbers} --- Sam Moore
- We plan to investigate the representation of real values to a high or arbitary degree of precision. Such representations would allow for the coordinates of primitives to be relative to a single global coordinate system. We would expect a decrease in performance with increased complexity of the data structure used to represent a real value. \rephrase{Both software and hardware techniques will be explored.} We will also consider the limitations imposed by performing calculations on the GPU or CPU.
+ We plan to investigate the representation of real values to a high or arbitary degree of precision. Such representations would allow for the coordinates of primitives to be relative to a single global coordinate system. We would expect a decrease in performance with increased complexity of the data structure used to represent a real value. We will also consider the limitations imposed by performing calculations on the GPU or CPU.
Starting points for research in this area are Priest's 1991 paper, ``Algorithms for Arbitrary Precision Floating Point Arithmetic''\cite{priest1991algorithms}, and Goldberg's 1992 paper ``The design of floating point data types''\cite{goldberg1992thedesign}. A more recent and comprehensive text book, ``Handbook of Floating Point Arithmetic''\cite{HFP}, published in 2010, has also been identified as highly relevant.
\item {\bf Local coordinate systems} --- David Gow \cite{proposalGow}
- An alternative approach involves segmenting the document into different regions using fixed precision floats to define primitives within each region. A quadtree or similar data structure could be employed to identify and render those regions currently visible in the document viewer.\rephrase{Say more here?}
+ An alternative approach involves segmenting the document into different regions using fixed precision floats to define primitives within each region. A quadtree or similar data structure could be employed to identify and render those regions currently visible in the document viewer.
\end{enumerate}
We aim to compare these and any additional implementations considered using the following metrics:
\section{Software and Hardware Requirements}
-Due to the relative immaturity and inconsistency of graphics drivers on mobile devices, our proof of concept will be developed for a conventional GNU/Linux desktop or laptop computer using OpenGL. However, the techniques explored could easily be extended to other platforms and libraries.
+Our proof of concept will be developed for a conventional GNU/Linux desktop or laptop computer using the OpenGL 3.1 API for rendering. However, the techniques explored could be extended to other platforms and libraries.
git.ucc.asn.au / ipdf / sam.git / commitdiff
UCC git Repository :: git.ucc.asn.au