1 Some implementation notes:
4 - We almost certainly want to have an "object" based system at some level,
5 if only because the types of things found in documents are of pretty different types
6 (text, embedded raster images, vector shapes, etc). For the most part, these are
7 also easily bounded spatially: probably by an axis-aligned bounding box. (A bounding
9 - By giving each object a "bounding box" of some sort (ideally one which also has extents in the "zoom"
10 direction, probably by having a min/max area, we can easily cull objects that will not be
12 - If objects can have children, and form a tree, we have a scene graph. The advantage here is
13 that when we cull an object, we don't need to process its children. It's also easy to implement
14 much of this with a matrix stack à la traditional OpenGL. On the downside, there's a growing view
15 that scene graphs are almost a pathologically bad case for CPU caches and minimizing GPU state changes.
16 See: http://home.comcast.net/~tom_forsyth/blog.wiki.html#[[Scene%20Graphs%20-%20just%20say%20no]] + others
18 - We could have a list of objects, but it would need to be traversed every time the view changes.
19 - Some sort of spatial data structure:
20 - Quadtree, kd-tree, bsp.
21 - Store a scenegraph, flatten on load?
22 - Lose culling info, probably.
23 - Giant array, cull on GPU w/ transform feedback and/or occlusion query?
24 - Current mobile hw probably can't do this, though we can fall back to
25 simpler methods if not available.
30 - Give each object a 'detail' parameter.
31 - We can then use higher/lower quality rendering for that object.
32 - More iterations of an algorithm, for example.
33 - Smoother Bézier curves.n
34 - Or lower resolution for embedded raster images
35 - Mipmaps let us upload a downscaled image to the GPU nicely.
36 - ARB_sparse_texture extension if you want to get fancy.
37 - Different rendering algorithm for complicated objects like font glyphs.
38 - Make it a function of zoom and (potentially) rendering state (below)
42 - There are basicaly three states we can be in when rendering:
43 - The view hasn't moved, isn't going to move
44 - We'll want to render everything at the highest possible quality and save the resultant
45 image, as it can the be reused.
46 - (Also render a little bit around the edge to help with the next case).
47 - The view has been translated, but no other transformation has occurred.
48 - We can reuse part of the current screen, if we're not translating too far.
49 - Things like font glyphs will be at the same size, so likely will not need reuploading.
50 - Any transformation has occurred.
51 - User wise, they'll be in the middle of a zoom.
52 - We could scale part of the existing screen, but that's what existing things do and it's ugly.
53 - Render at a lower quality. (Signed-distance-field font rendering and/or glyphy).
54 - Simple hereustic for mobile/touchscreen: do this if there are two+ fingers on the screen.
57 - A given object at a given resolution can be easily cached in an FBO
58 - Better still, if something is easily represented as geometry, we can just
59 store a VBO/VAO with the rendering requirements for that object (at that LOD),
60 and simply set exact position/scaling as uniforms. All the imuportant data will
61 be on the GPU already.
62 - Simple LRU of cached objects.
63 - Or a pair of caches: one for long-term objects at current quality, one for things
64 used during a zoom that can be flushed quickly when the zoom has been completed.
65 - Keep a low-quality cache of most objects to include while the real, high quality one is
67 - How many frames latency can we have before the high-quality "correct" version is displayed.
68 - Is it worth making sure all objects hit the high-quality versions at the same time?
71 - Objects can have themselves as children.
72 - Scene "tree" -> directed Scene graph.
73 - Objects are culled based on size, so we don't end up with runaway recursion
74 when objects' children are smaller than object.
75 - Can't do arb. fractals.
76 - Unless we allow objects to be rotated (possible), we can only do axis-aligned
78 - Have to flatten this out each frame, could potentially get a bit slow.
79 - A "shader" object, which applies an arb. shader.
80 - Pass quality/resolution to shader as a uniform.
81 - Pixel/fragment shaders scale nicely, can do mandelbrot set and friends easily.
82 - Things like Koch snowflake could be done with Geom shaders
83 - possibly with transform feedback for a simple, recursive approach.
84 - or loops, but that could get more complicated as we need to do things in a
86 - No geom shaders on mobile, but could potentially emulate on CPU.
87 - Security implications if done naïvely.
88 - Custom bytecode/language that is "compiled" into GLSL or interpreted on CPU
90 - Could JIT if really keen.
91 - Don't make it turing complete, only allow loops for either a constant
92 number of iterations or a number of iterations proportional to the "quality".
95 - We'll probably want to take advantage of multi-core CPUs.
96 - One "GPU" thread w/ OpenGL context, basically pulling commands off a FIFO and
98 - Or pointers to "command buffers", each with an array of commands.
99 - Use ARB_map_buffer_range/ARB_buffer_storage to map GPU memory for use by other threads.
100 - Then can use fences/memory barriers to handle sync?
101 - Threads which take subtrees/subgraphs and build command queues for rendering them.
102 - Thread for font rasterization/misc data loading.