Some implementation notes: Objects: - We almost certainly want to have an "object" based system at some level, if only because the types of things found in documents are of pretty different types (text, embedded raster images, vector shapes, etc). For the most part, these are also easily bounded spatially: probably by an axis-aligned bounding box. (A bounding rectange). - By giving each object a "bounding box" of some sort (ideally one which also has extents in the "zoom" direction, probably by having a min/max area, we can easily cull objects that will not be visible. - If objects can have children, and form a tree, we have a scene graph. The advantage here is that when we cull an object, we don't need to process its children. It's also easy to implement much of this with a matrix stack à la traditional OpenGL. On the downside, there's a growing view that scene graphs are almost a pathologically bad case for CPU caches and minimizing GPU state changes. See: http://home.comcast.net/~tom_forsyth/blog.wiki.html#[[Scene%20Graphs%20-%20just%20say%20no]] + others I'll dig up. - We could have a list of objects, but it would need to be traversed every time the view changes. - Some sort of spatial data structure: - Quadtree, kd-tree, bsp. - Store a scenegraph, flatten on load? - Lose culling info, probably. - Giant array, cull on GPU w/ transform feedback and/or occlusion query? - Current mobile hw probably can't do this, though we can fall back to simpler methods if not available. - LOD: - Give each object a 'detail' parameter. - We can then use higher/lower quality rendering for that object. - More iterations of an algorithm, for example. - Smoother Bézier curves.n - Or lower resolution for embedded raster images - Mipmaps let us upload a downscaled image to the GPU nicely. - ARB_sparse_texture extension if you want to get fancy. - Different rendering algorithm for complicated objects like font glyphs. - Make it a function of zoom and (potentially) rendering state (below) The view: - There are basicaly three states we can be in when rendering: - The view hasn't moved, isn't going to move - We'll want to render everything at the highest possible quality and save the resultant image, as it can the be reused. - (Also render a little bit around the edge to help with the next case). - The view has been translated, but no other transformation has occurred. - We can reuse part of the current screen, if we're not translating too far. - Things like font glyphs will be at the same size, so likely will not need reuploading. - Any transformation has occurred. - User wise, they'll be in the middle of a zoom. - We could scale part of the existing screen, but that's what existing things do and it's ugly. - Render at a lower quality. (Signed-distance-field font rendering and/or glyphy). - Simple hereustic for mobile/touchscreen: do this if there are two+ fingers on the screen. GPU Data caching: - A given object at a given resolution can be easily cached in an FBO - Better still, if something is easily represented as geometry, we can just store a VBO/VAO with the rendering requirements for that object (at that LOD), and simply set exact position/scaling as uniforms. All the imuportant data will be on the GPU already. - Simple LRU of cached objects. - Or a pair of caches: one for long-term objects at current quality, one for things used during a zoom that can be flushed quickly when the zoom has been completed. - Keep a low-quality cache of most objects to include while the real, high quality one is still being computed? - How many frames latency can we have before the high-quality "correct" version is displayed. - Is it worth making sure all objects hit the high-quality versions at the same time? "Fractal" structure: - Objects can have themselves as children. - Scene "tree" -> directed Scene graph. - Objects are culled based on size, so we don't end up with runaway recursion when objects' children are smaller than object. - Can't do arb. fractals. - Unless we allow objects to be rotated (possible), we can only do axis-aligned things. - Have to flatten this out each frame, could potentially get a bit slow. - A "shader" object, which applies an arb. shader. - Pass quality/resolution to shader as a uniform. - Pixel/fragment shaders scale nicely, can do mandelbrot set and friends easily. - Things like Koch snowflake could be done with Geom shaders - possibly with transform feedback for a simple, recursive approach. - or loops, but that could get more complicated as we need to do things in a specific order... - No geom shaders on mobile, but could potentially emulate on CPU. - Security implications if done naïvely. - Custom bytecode/language that is "compiled" into GLSL or interpreted on CPU if HW insufficient. - Could JIT if really keen. - Don't make it turing complete, only allow loops for either a constant number of iterations or a number of iterations proportional to the "quality". Threading: - We'll probably want to take advantage of multi-core CPUs. - One "GPU" thread w/ OpenGL context, basically pulling commands off a FIFO and executing them. - Or pointers to "command buffers", each with an array of commands. - Use ARB_map_buffer_range/ARB_buffer_storage to map GPU memory for use by other threads. - Then can use fences/memory barriers to handle sync? - Threads which take subtrees/subgraphs and build command queues for rendering them. - Thread for font rasterization/misc data loading.