20:40 < matches> I shall resume my epic battle with primary school level mathematics tomorrow
--- Day changed Sun Jul 06 2014
00:32 <@sulix> long division of an arbint by a uint64 worksish.
+09:47 <@sulix> long division of an arbint by a uint64 now actually works and is less obviously badly designed
+10:32 < matches> Thanks...
+10:33 < matches> In other news I seem to have broken X again
+10:34 <@sulix> Oh no!
+10:35 < matches> We need an ASCI renderer...
+10:36 <@sulix> That can not possibly end badly.
+10:40 < matches> shouldn't be too hard...
+10:40 < matches> Lets see if reinstalling fglrx works
+10:41 < matches> I need to add that to a boot script :S
+10:42 < matches> uhoh installation failed
+10:48 <@sulix> Hmm... enabling optimization makes Arbint segfault.
+10:49 < matches> You should probably make quadtrees
+10:50 < matches> Although I admit you're better at Arbints than me :P
+10:50 <@sulix> Yeah... I should.
+10:50 <@sulix> But segfaults...
+10:51 < matches> Yeah somewhere in std vector I had some as well
+10:51 < matches> I think I might change to a custom dynamic array
+10:52 < matches> Then fast copying is easier
+10:52 <@sulix> Yeah, it's segfaulting in std::swap somewhere, I think.
+10:52 < matches> I'll fix it when I have X
+10:53 < matches> Or an ascii renderer...
+10:54 < matches> Probably more realistic at this stage
+10:55 <@sulix> I'm pretty certain there's a unicode character for "face of disapproval due to fglrx bugginess".
+10:56 < matches> Yeah but when you try and render it you get a seg fault
+11:24 < matches> Woo, xvesa
+11:26 < matches> I see what you mean about fglrx being hard to remove
+11:58 <@sulix> So: quick profile. 98% of frame rendering time is spent in Arbint::Division.
+11:59 < matches> Yeah
+12:00 < matches> I *was* trying to install kcachegrind you know
+12:00 <@sulix> The vast majority of that time is spent in malloc and free.
+12:01 < matches> Well copy constructing an Arbint requires copy constructing the vector
+12:01 < matches> I'll investigate
+12:01 < matches> Now I have X again
+12:01 < matches> Go make a quad tree
+12:02 <@sulix> Also, should add_digits clear the carry flag at the beginning?
+12:02 < matches> Hmm
+12:02 < matches> Probably
+12:02 < matches> Yeah
+12:09 < matches> I suspect having a copy-on-write Arbint won't help as much as having a non-terrible division algorithm so I'll try fix that first
+12:09 < matches> But copy-on-write Arbint shouldn't be too hard
+12:09 < matches> Oh unless the original changes
+12:10 < matches> Yeah it would be hard
+12:10 < matches> I'm sure I did something like it before
+12:10 < matches> Back when I actually made my own data structures instead of just using std::vector
+12:15 <@sulix> My custom vector class: http://davidgow.net/stuff/DynamicArray.h
+12:15 <@sulix> Probably won't compile without a bunch of other files, though.
+12:16 < matches> I think I have several, in different versions of neurosis
+12:16 < matches> Which was never under git
+12:16 < matches> Some of them were prone to segfaults though
+12:17 < matches> So, will you have some kind of quad-tree-y thing by monday? :P
+12:17 <@sulix> Mine works, but doesn't call constructors or destructors, so it's only really useful for ints/floats/pointers or structs thereof.
+12:18 <@sulix> I have a "struct QuadTree".
+12:18 <@sulix> (You can't do anything with it, but the basic idea is there)
+12:20 < matches> My shifting operators might not have been as fith as I thought last night
+12:20 < matches> Since 7 and e do actually have the same number of bits
+12:21 < matches> I'm not sure where that came from
+12:21 < matches> Ah dammit
+12:21 < matches> Cannot find -lGL
+12:22 <@sulix> Try just removing that from the Makefile
+12:22 <@sulix> I think when I GL-3'd everything I made it load libGL at runtime.
+12:23 < matches> Well it will compile but I get glx errors
+12:23 < matches> I get similar errors running glxinfo though...
+12:24 < matches> Odd
+12:26 <@sulix> I think you still have some of fglrx breaking things.
+12:26 <@sulix> It replaces the system libGL.so
+12:26 < matches> I installed mesa's stuff
+12:27 <@sulix> You need to also get the version of libGL that runs as part of the X server, IIRC.
+12:28 < matches> I'll try some apt-get --reinstalls
+12:29 < matches> I think most of my problems were that debian was going "You don't have a graphics driver, here, have one" and then things were breaking and reinstalling fglrx would give me graphics but leave me with a partially installed debian driver as well :S
+12:30 < matches> Well ok
+12:30 < matches> most of my problems were fglrx
+12:30 <@sulix> I remain sceptical that installing fglrx by hand will ever cause anything that is not pain.
+12:30 < matches> Well it worked better than the open source drivers for a very long time
+12:30 < matches> It all ended in tears and segfaults though
+12:32 < matches> Also it worked better than the non-free fglrx package in the debian repos which always ended in tears and segfaults
+12:33 <@sulix> I use the non-free nvidia package, which works fine, even if it is horribly outdated.
+12:34 <@sulix> I'm scared of the sheer amount of ripping up the rendering code getting quadtrees working will take.
+12:39 < matches> Haha
+12:40 < matches> It might be easier to do the CPU rendering in quad trees first?
+12:40 < matches> I'm not quite sure how you'll do it
+12:41 < matches> CPU rendering at least gives you direct control over the rendering target as a pixel buffer
+12:41 < matches> GPU rendering is a bit shady (ho ho)
+12:41 <@sulix> ...
+12:42 <@sulix> That was terrible, yet accurate!
+12:42 < matches> With the CPU rendering we're already passing the objects list, view, and target bounds
+12:42 < matches> So maybe you can adapt View or something to be quad-tree-ish
+12:43 < matches> Good luck
+12:44 <@sulix> Yeah, the current plan is to adapt the rendering stuff to accept a "first object id" and "last object id", and then only render those.
+12:47 < matches> Hmm, you could call RenderUsingCPU with a different target pixel buffer for each part of your quad tree or something
+12:48 < matches> Oh well, hopefully I can adapt whatever you do into RenderUsingASCII :P
+12:49 < matches> I don't need GLX working to try and fix division but I'd really like to have GLX working
+12:49 < matches> Sigh
+13:12 < matches> I'm making progress, instead of reporting an invalid operation glxinfo segfaults now
+13:13 < matches> So I have the wrong libraries or something... how to get the right ones...
+13:32 <@sulix> Woo. I got it compiling again!
+13:32 <@sulix> And the rendering is only half-wrong.
+13:33 < matches> Nice
+13:33 < matches> Manually deleting anything on my system with fglrx in it...
+13:34 <@sulix> Someone really needs to write an "fglrx removal guide".
+13:34 < matches> I'm not sure why reinstalling the radeon driver and mesa isn't actually giving me a libGL
+13:34 < matches> At least
+13:34 < matches> I suspect it is just leaving the existing broken one
+13:36 <@sulix> So it turns out that *2 I don't understand is important.
+14:01 < matches> Ok
+14:01 < matches> I have glxgears
+14:01 < matches> An epic achievement
+14:01 < matches> I had to go back to fglrx but at least this time its the debian packaged version
+14:02 < matches> The radeon driver seems to work better for stuff that isn't glx
+14:03 < matches> Which is pretty cool but I kind of like glx actually working
+14:11 < matches> Ok... time to fix division
+14:11 < matches> Finally
+14:11 < matches> No wait time for coffee
+14:11 < matches> Then time to fix division
+14:24 < matches> Did you test the div_digits function at all?
+14:25 < matches> # We want to point to the end of the buffer (LSB)
+14:25 < matches> Aren't I doing it the other way around...
+14:25 < matches> I'll test it
+14:28 <@sulix> With long division you do things in the opposite direction to addition.
+14:29 <@sulix> I tested it with some simple things and it worked.
+14:34 < matches> Hmm
+16:35 < matches> Well the good news is that division is faster now
+16:35 <@sulix> Cool!
+16:36 < matches> The bad news is that I can't see anything
+16:36 < matches> Even when GPU rendering
+16:36 < matches> Also segfaults
+16:36 <@sulix> I've just pushed some initial quadtree-enabling breakage.
+16:36 < matches> I didn't work out how to apply the div_digits to a multi digit number either
+16:36 <@sulix> Yeah, I thought about that, then went to bed.
+16:36 < matches> I might convert the current algorithm which is all based on bit shifting
+16:36 < matches> To assembly
+16:37 < matches> But maybe not
+16:37 < matches> Actually being able to see something render would be good even if it were slow
+16:37 < matches> Then I can think about making it faster
+16:38 <@sulix> The old one had all of the beziers connected for no readily apparent reason.
+16:38 < matches> I have a tester called "Arbitrary Rationals" (arbrationals.cpp) which sounds like some kind of political party
+16:39 < matches> A rather indecisive political party
+16:39 <@sulix> That's amazing.
+16:48 < matches> Ah, I might be having trouble with the sign of things, hm
+16:49 < matches> Rational 15625/288230376151727369 (0.000000) is not close enough at representing 1.000000 (1.000000 vs 0.001000)
+16:49 < matches> That doesn't look like a sign problem...
+16:52 <@sulix> I admit, 15625/288230376151727369 is not really similar to 1.
+16:54 < matches> So, somewhere along the way Arbint(1e6) became 15625 and Arbint(1e6) also became 288230376151727369 ?
+16:54 < matches> Uninitialised things?
+16:54 < matches> Constructing from int64_t seems fine
+17:00 < matches> That amazing moment when it works perfectly and then you realise that's because you compiled with reals as doubles and not rationals
+17:00 <@sulix> Ha: I've done that a few times.
+17:19 < matches> Maybe I shouldn't have pulled your latest commit
+17:19 < matches> Now I have two problems :(
+17:22 < matches> Meeting tomorrow
+17:22 < matches> "So how much progress have you made?"
+17:22 < matches> "Segfaults are up 100% to a record high!"
+17:26 < matches> Oh ok
+17:26 < matches> I'm not sure if this is new
+17:26 < matches> But if there is nothing in the document the GPU rendering doesn't like it
+17:27 <@sulix> Hmmm... there might be some issues with that.
+17:28 < matches> Ok
+17:28 < matches> Rectangle renders
+17:28 < matches> With arbitrary rationals
+17:28 < matches> In the default view
+17:29 <@sulix> Ooh!
+17:31 < matches> I'm going to commit this even though it's totally broken
+17:32 <@sulix> Things being broken hasn't stopped me from committing them.
+17:35 < matches> I love this though
+17:35 < matches> When you activate CPU rendering
+17:36 < matches> You get like a billion errors from Rational<Arbint>::CheckAccuracy
+17:36 < matches> And then the Bezier appears!
+17:36 < matches> Magically!
+17:36 < matches> I guess I shouldn't rule out a bug in CheckAccuracy :P
+17:36 < matches> Although zooming will definitely cause a segfault
+17:38 <@sulix> I tried running the really slow version under valgrind, but that turned out to be a bad idea.
+17:39 < matches> I've pushed the slightly less really slow version
+17:39 < matches> That is still really slow
+17:39 < matches> I can't install kcachegrind
+17:39 < matches> It is really annoying
+17:40 < matches> There aren't any valgrind analysis things that don't require the entirity of KDE are there?
+17:40 < matches> Or should I just learn how to read the raw output files
+17:40 <@sulix> Cool: it works.
+17:41 <@sulix> (I dunno, I just use KDE :P)
+17:41 <@sulix> You can try "perf" or "oprofile".
+17:41 < matches> Well I can't even install KDE :S
+17:41 <@sulix> They're terminal-based.
+17:41 <@sulix> Well, perf is, at least. There's a gtk+ oprofile gui.
+17:41 <@sulix> They both measure the entire system by default though.
+17:43 <@sulix> Rational::Simplify is ~54% of time (w/ GPU rendering) with the new build, according to callgrind.
+17:50 < matches> I'm a little confused by some of the outputs of Rational::CheckAccuracy given that it does actually seem to render the bezier properly on the CPU
+17:50 < matches> I think the segfault might be related to my (ab)use of things like memcpy on std::vector::data()
+17:52 < matches> Also I think I've fixed my dependency hell
+17:52 < matches> Well
+17:53 < matches> Depending on how this turns out I will end up with the entirity of KDE hopefully including kcachegrind, or I will end up with no X again
+17:53 < matches> And much swearing will ensue
+17:54 <@sulix> Yeah, it segfaults immediately if you enable optimization level 2 or higher.
+17:55 < matches> Hm
+17:56 < matches> I'll probably get rid of std::vector then
+17:57 <@sulix> The problem I'm seeing is in the vector<Bezier>::push_back() bit, so who knows what is going on.
+17:58 < matches> Oh that's odd
+17:58 <@sulix> Okay, don't run it in valgrind.
+17:58 <@sulix> I'm getting about 200 "uninitialized values" a second in the nVidia driver.
+17:58 < matches> But I just installed kcachegrind!
+17:59 <@sulix> (Oh wait, that was with REAL=double)
+17:59 < matches> Haha
+18:00 < matches> Wait, vector<Bezier>::push_back causes problems even with REAL=double :S
+18:00 <@sulix> Nah, the nVidia driver seems to.
+18:01 <@sulix> I've now compiled again with REAL=5 and I'm getting a number of invalid reads off the end of the vector<digit_t> in GetBit()
+18:02 < matches> Division
+18:02 < matches> Oh
+18:02 < matches> oooh
+18:02 < matches> > -> >=
+18:02 < matches> ...
+18:03 <@sulix> Well, my first attempt to fix it just broke rendering.
+18:04 <@sulix> Ah, yep, I see it.
+18:05 < matches> I think I need to do less memcpy/memseting and more std::vector::operator= -_-
+18:05 < matches> I mean, they implemented it for a reason
+18:05 <@sulix> So, with that change it no longer segfaults!
+18:05 <@sulix> (It corrupts memory and throws an exception instead)
+18:05 < matches> Woah
+18:06 < matches> I changed Arbint operator= to use std::vector operator=
+18:06 < matches> And now I get a filled circle on top of my bezier...
+18:07 < matches> Ok, I will rewrite Arbint to just rely solely on std::vector operations
+18:08 < matches> That might slow it down but it's impossible to test that the maths works properly with all these memory errors
+18:09 <@sulix> Should you be using m_sign more often?
+18:09 <@sulix> It seems to be "forgotten" in a number of places...
+18:10 <@sulix> Hmm... everything looks like it's working at the moment, actually.
+18:10 < matches> What else did you change?
+18:10 < matches> Try zooming
+18:11 < matches> That will break it
+18:11 <@sulix> Zooming seemed to work, actually.
+18:11 < matches> Are you sure you compiled with REAL=5
+18:11 < matches> The sign is a bit wierd
+18:12 <@sulix> It works for a little bit, then the view bounds suddenly become huge.
+18:12 < matches> Haha
+18:12 <@sulix> For no reason, the y coordinate jumps to 10^8.
+18:13 <@sulix> Also after you've zoomed, sometimes translating goes in the wrong direction or is otherwise broken.
+18:15 < matches> That might be related to m_sign...
+18:15 < matches> The joys of implementing your own number representation
+18:16 <@sulix> Trying to "fix" the sign fixed zoom, but broke translation even more.
+18:18 <@sulix> Also: best representation of 1 ever: 9223372036854775807/9223372036854775807
+18:18 < matches> Hahaha
+18:18 < matches> Oh
+18:18 < matches> I think I commented out a Simplify() somewhere :P
+18:18 < matches> No
+18:18 < matches> I didn't
+18:18 < matches> Hmm
+18:19 < matches> Well I am going to push some minor changes
+18:20 <@sulix> Also: T g = gcd(T(llabs(P)),T(llabs(Q)));
+18:20 <@sulix> Is that "llabs" truncating everything?
+18:20 < matches> ...
+18:20 < matches> Probably
+18:20 < matches> !
+19:05 <@sulix> Well, my attempt to write a templated abs() function that worked for Arbint may have worked, but it crashed X before I could find out.
+19:05 <@sulix> I do not want to know how that is possible.
+19:05 < matches> Heh, I was trying to do that before dinner
+19:06 < matches> I just got segfaults
+19:08 < matches> Oh
+19:08 < matches> I see
+19:08 <@sulix> I suspect the best thing for me to do is to just stash all of my changes and break some more quadtrees.
+19:08 < matches> Yeah
+19:08 <@sulix> But first I'll have to get X working again.
+19:08 < matches> The problem with overriding llabs is that the constructor for Arbint calls llabs...
+19:08 < matches> And Arbint has a constructor from int64_t...
+19:09 < matches> So as soon as you implement llabs using Arbint it will call itself for eternity
+19:09 <@sulix> I made a new function called "real_abs" (because "i_went_to_the_gym_now_look_at_my_abs" was too long) and just used that.
+19:09 < matches> Bahaha
+19:10 <@sulix> Then X started using 100% of my CPU and nothing responds to anything.
+19:10 < matches> :S
+19:10 <@sulix> The mouse cursor still moves, but everything else is a black screen.
+19:11 <@sulix> Excellent, "sudo pkill -9 Xorg" has made everything work again.
+19:12 <@sulix> (And even fixed an unrelated problem, which is good but worrying)
+19:17 <@sulix> Note to self: if something crashes X when you run it once, don't run it again after fixing X.
+19:18 < matches> Haha
+19:18 <@sulix> Now to try it for a third time.
+19:18 < matches> I've got an abs working
+19:18 <@sulix> Ah, excellent.
+19:19 <@sulix> Does it make things better? Worse? The same but slower?
+19:19 <@sulix> Crash X?
+19:20 < matches> It doesn't crash X but I think I have an infinite loop in gcd now
+19:21 < matches> I should probably work out why X/X for really big X is not simplifying to 1/1
+19:43 < matches> So it turns out it's a pretty good idea to have a tester that actually tests lots of different numbers with every single operation
+19:43 < matches> As opposed to
+19:43 < matches> "1/2 == 0.5" yep looks good she'll be right
+19:43 < matches> One could say
+19:43 < matches> There are *signs* of serious problems
+19:44 * matches puts on sunglasses
+19:44 < matches> No wait I got the order wrong
+19:44 < matches> Oh well I wasn't cool enough to watch that show anyway
+20:18 < matches> So now I have a tester that actually tests things
+20:18 < matches> I just need to blindly fumble around with m_sign until all the tests are passed!
+20:18 < matches> Right?!
+20:26 < matches> Heh, it's amazing how many errors fixing the sign in Division prevents
+20:27 < matches> Well amazing until you realise that every single Rational calls a division at least once
+20:44 <@sulix> Hmmm... there seem to be a lot of cases where we get things becoming 1 (randomnumber/randomnumber) instead of what they should be, according to CheckAccuracy.
+20:46 < matches> Yeah
+20:46 < matches> It should work nicer nowish
+20:46 < matches> At least, the tester works most of the time...
+20:46 < matches> It still randomly fails sometimes
+20:46 <@sulix> Yeah, it's working much better.
+20:46 < matches> Now 12% of the time is in std::vector::size()
+20:46 <@sulix> All of the things I've seen have been to do with subtraction in ScaleAroundPoint()
+20:47 < matches> Ah
+20:47 < matches> Wait I haven't pushed the 12% in std::vector::size yet
+20:47 < matches> tl;dr I was allocating one extra digit every single addition operation and then deleting it if it wasn't used
+20:48 < matches> Because I am just that good :P
+20:48 < matches> I suspect the Arbint is going to break when it actually does need to resize itself
+20:49 < matches> But for now I will settle for "At least as good as an int64_t"
+20:49 < matches> * But slower
+20:49 <@sulix> I'm curious as to why digit_t is int64_t and not uint64_t.
+20:49 < matches> Oh it is uint64_t now
+20:49 < matches> It was int64_t mainly because geany doesn't syntax highlight uint64_t
+20:50 <@sulix> Ah.
+20:50 < matches> Then I realised being int64_t instead of uint64_t broke the shift operators
+20:50 < matches> Because they try to preserve the sign
+20:50 <@sulix> Shift on signed ints is undefined behaviour, IIRC.
+20:51 < matches> Yeah I was considering writing the shift in assembly but its easier to handle the underflow/overflow into the next digit in C
+20:52 < matches> Zooming is still incredibly slow
+20:54 < matches> In fact it is infinitely slow
+20:54 <@sulix> I once waited about 10 minutes and did get another frame.
+20:54 < matches> In fact the Arbint appears to want to grow an infinite number of 0L digits
+20:55 <@sulix> Ah ha! I have realised why CheckAccuracy is only throwing errors on "-".
+20:55 <@sulix> The CheckAccuracy calls for everything else have been commented out,
+20:55 < matches> They should all be commented out so I can use tests/realops.test instead
+20:57 < matches> 23 / 12000 operations failed... doesn't seem too bad...
+20:58 <@sulix> Keep in mind that == isn't precise as it casts to doubles.
+20:58 < matches> You get more like 40 out of Rational<int64_t>
+20:58 < matches> Yeah they are "equal" if it is within 1e-1 :S
+20:58 < matches> I'm searching for the more totally catastrophic failures like the wrong sign first :P
+21:20 < matches> Ok tomorrow I might just try using boost arbitrary integers or something
+21:28 <@sulix> All of the failures I'm getting are with "/="
+21:28 <@sulix> Except one, which was only just outside the error bounds, so probably the doubles' fault.
+21:30 < matches> I was restricting the test values somewhat though
+21:30 < matches> (With rand()%100 + 1)
+21:33 <@sulix> You do know that the Rational<Arbint> -> double conversion is totally broken if either P or Q have > 1 digit.
+21:33 < matches> Yeah I noticed that
+21:37 <@sulix> Oh man, re-enabling the operator double on Arbint totally breaks things.
+21:40 < matches> Yeah don't touch it
+21:40 < matches> Basically touching anything will totally break something else
+21:40 < matches> :S
+21:40 < matches> I should probably sleep some more before attempting to fix things
+21:40 <@sulix> The other bug I just found is that Arbint didn't have the unary "-" operator, so writing "-P" would silently cast to int64_t.
+21:41 < matches> Ah ok
+21:41 < matches> I thought unary - was equivelant to "0 - x" but I suppose that doesn't make sense
+21:41 <@sulix> Nah, it has to be written explicitly.
+21:44 <@sulix> Okay, I think I've got realops.test down to 0 errors with Arbint.
+21:45 < matches> Well you're doing better than me
+21:46 <@sulix> I'm running it with a huge number of tests now.
+21:47 <@sulix> 85.95% of my entire computer's processing time is in IPDF::Rational<IPDF::Arbint>::operator/=
+21:48 < matches> Can you push what you have?
+21:48 <@sulix> Yeah, I will in a second.
+21:48 <@sulix> It's basically a lot of hacks.
+21:49 <@sulix> CPU rendering is broken, though.
+21:49 < matches> There were cases where it would grow an infinite number of digits; did you fix those?
+21:49 <@sulix> Probably not: it still gets very, very slow zooming.
+21:51 < matches> Yeah there's something wrongish it shouldn't need extra digits so fast
+21:51 <@sulix> Pushed now.
+21:52 <@sulix> Now to look at the diff and see what I actually changed.
+21:53 < matches> Oh
+21:54 < matches> Simplify call in Rational operator= is good
+21:56 < matches> I think I fixed some more bugs but I also introduced an infinite loop so...
+21:57 < matches> How is the quad tree bit going :P
+21:57 <@sulix> Yeah, about that...
+21:58 < matches> Remember Arbints don't have to work for quad trees to work :P
+21:58 < matches> Or do they
+21:58 <@sulix> In theory, no.
+21:58 <@sulix> In practise, it probably depends on how one defines "success".
+21:59 < matches> I think we need to design a test pattern where you actually want to zoom forever
+21:59 <@sulix> Highly theoretically, I'm pretty certain the QuadTree basically ends up isomorphic to the Arbint.
+21:59 < matches> Seems legit
+21:59 < matches> Can you prove it?
+21:59 < matches> That would be a useful result
+21:59 <@sulix> Yes*
+22:00 <@sulix> *hopefully
+22:00 < matches> I'm afraid most of my "theory" is based solely on intuition at this point :S
+22:00 < matches> Well the division algorithm came from wikipedia but division isn't as intuitive as the other operations
+22:01 < matches> Actually if I can fix all these other stupid bugs I'll try my binary searchish division idea
+22:01 <@sulix> Yeah, I spent a while racking my brain for long division, and then only managing to work it out for dividing by a single digit.
+22:02 <@sulix> That's really, really easy in asm, as well, which is nice, but not quite as useful as I'd hoped.
+22:02 < matches> It's pretty useful; we can still have a "if div.m_digits.size() use the assembly function"
+22:02 < matches> * size() == 1 that is
+22:02 <@sulix> Wikipedia mentioned doing newton-raphson for division, which is a terrifying thought.
+22:02 < matches> Haha
+22:03 < matches> So I've got Arbint::Shrink() calls pretty much after every operation now for no apparent reason
+22:04 <@sulix> Does it help?
+22:05 < matches> On the 551st test Shrink has been called 4 times in a row and then something after that causes an infinite loop
+22:05 < matches> I can't work out what it is
+22:05 < matches> I have a GrowDigit that spams Debug messages now
+22:05 < matches> So its not that
+22:05 < matches> It's silent
+22:05 < matches> Let's see what callgrind has to say
+22:06 < matches> A few minutes should be enough for it to be obvious right
+22:07 < matches> I think the division loop runs forever
+22:07 < matches> Bugger
+22:08 < matches> Wait that doesn't make sense there's only 1 digit
+22:08 < matches> Blargh
+22:17 <@sulix> So using the asm version when possible makes realops.test 20x faster, but there were a couple of failed tests.
+22:18 <@sulix> I'm not sure if they're new or not, because I was able to run many more tests.
+22:20 <@sulix> Also, I think I've found your infinite loop.
+22:23 <@sulix> Okay, maybe I've just obfuscated the code some more.
+22:24 < matches> Infinite loop is due to subtraction bug
+22:24 < matches> I'm pretty sure
+22:24 <@sulix> Yeah, pushing a "fix"
+22:26 <@sulix> Pushed, but think I might have found a bug in the fix.
+22:26 < matches> Ok
+22:27 < matches> (0,0) - +(16292953875657448384,1) = -(16292953875657448384,2) seems wrong
+22:28 <@sulix> Pish! Nothing could be more beautiful.
+22:28 < matches> It's pretty hard to debug with such huge numbers :P
+22:31 <@sulix> Try this...
+22:31 * sulix pushes.
+22:32 < matches> What does that do...
+22:33 < matches> oh
+22:33 < matches> You negate the entire thing and add one
+22:33 < matches> Seems legit
+22:33 <@sulix> So "two's complement" negatives mean that -A == flip all bits of A and add 1.
+22:33 < matches> Yeah
+22:34 <@sulix> There seems to be a bug in the asm div_digits though.
+22:35 < matches> I'm taking that out until I know the main algorithm works :P
+22:35 < matches> Silly I spent so much time wondering why division was broken without checking subtraction
+22:36 < matches> After I spent all that time thinking I was so clever for doing division using bit shifts and subtractions
+22:37 <@sulix> It still takes forever.
+22:37 <@sulix> (Zooming in, that is)
+22:37 < matches> Nah there's still a bug
+22:37 < matches> You're running with an older version of realops test
+22:37 < matches> Hang on
+22:37 < matches> I will push things
+22:38 < matches> Trying to avoid a merge...
+22:38 <@sulix> And I will get a drink.
+22:44 < matches> So if you run enough tests you eventually get a bunch of SubBasic spam
+22:45 < matches> Which is actually occuring due to "<" operators
+22:45 <@sulix> Yeah: I'm going to add a bunch of hacks to "<" at some point.
+22:45 < matches> Which are in the Division loop ( >= is implemented in terms of <)
+22:45 < matches> Don't add the hacks until subtraction works!
+22:46 < matches> I don't want to optimise it yet
+22:46 < matches> Otherwise all the optimisations hide the bugs
+22:47 < matches> Time for a tester that just does subtractions I think
+22:48 <@sulix> It might be worth adding a bunch of SDL_assert_paranoid()'s everywhere that check that (a - b)+b == a and similar on every - call.
+22:49 < matches> Most of the tested numbers can be represented in a single digit which is causing problems
+22:50 < matches> With finding the bugs in the multi-digit algorithm
+22:52 < matches> For small enough multi digit Arbints testing against doubles might work?
+22:52 <@sulix> Maybe.
+22:53 <@sulix> You'd lose enough precision that it's not worth it for anything > 2 digits, I'd say.
+22:53 < matches> Yeah, 2 digits should be enough to make sure the algorithms actually work though
+22:54 < matches> 2 digits in one operand and 1 in the other (which will resize itself to have a leading zero if necessary)
+22:55 < matches> At some point I am going to have to use a library for Arbints and compare ours to it
+22:55 < matches> But I kind of want ours to work properly before bringing in an external one
+22:56 < matches> When I try floats I might just bring in the external library :P
+22:56 <@sulix> Two questions:
+22:57 <@sulix> 1. What is "borrow" supposed to be? Is it the "carry" or the "result has changed sign" or something else?
+22:57 <@sulix> 2. What do you think of replacing Sub() with Add() + Negate()?
+22:59 < matches> borrow's like carry (I'm using subtract with borrow in asm) it should get subtracted from the next digit, and if at the end it is still set then it means your result is less than zero
+22:59 < matches> and 2 sounds like an excellent idea
+23:00 <@sulix> Okay.
+23:00 <@sulix> 'cause there's a separate "carry" flag and "sign" flag, so I wasn't sure.
+23:01 < matches> subtract actually working would be slightly faster than add and then negate I think
+23:01 <@sulix> I'm not 100% sure if all the code is using m_sign as the sign or if some of it is trying to use 2's complement, which isn't really possible with arbints.
+23:03 < matches> When I was testing the subtraction code I was using int64_t instead of uint64_t and it seemed to work :S
+23:04 < matches> I should check whether sbb does unsigned or signed subtraction
+23:04 <@sulix> Both, I think.
+23:04 <@sulix> 2's complement makes them the same, which is why it's so nice.
+23:04 <@sulix> But it requires a fixed number of bits to work.
+23:09 < matches> Whoops now the bit shift tester runs forever as well
+23:13 < matches> Oh no it doesn't
+23:13 < matches> It just exceeds the limit on GrowDigits
+23:14 < matches> Phew
+23:14 < matches> Bit shifting works at least
+23:14 < matches> Or at least it is consistently broken
+23:14 < matches> (a >> X) << X == a
+23:14 < matches> But == could be broken
+23:14 < matches> NOTHING IS SAFE
+23:14 < matches> When you are writing your own number represntation :(
+23:15 <@sulix> Idea: 2's complement subtraction will make -ve numbers have an "infinite" number of 1s at the front, so would that be infinite looping, trying to continually add digits with 1s.
+23:15 < matches> Operator overloading is amazingly powerful and yet it seems like the best way to make seemingly sane code do batshit insane things
+23:15 <@sulix> I think the answer is "no: we're always trapping it and converting it" but am not sure.
+23:16 < matches> Our subtraction doesn't take forever
+23:16 < matches> The issue is when the result would be less than zero
+23:16 < matches> Hang on let me make another tester
+23:17 < matches> Well, I'll put it in the arbint tester instead of the real tester
+23:18 < matches> Since it's pretty clear the bugs are in arbint not rational<arbint> itself
+23:18 < matches> Although there might be bugs in rational<T> as well
+23:18 < matches> But we'll squish those when we get to them
+23:18 < matches> Also quad trees are they a thing yet?
+23:20 <@sulix> "The quadtrees aren't in the code, they were in your heart all along."
+23:21 < matches> Haha
+23:21 < matches> Tell that to Tim :P
+23:21 < matches> Ok (45,10) - (128,0) = -(83,18446744073709551605)
+23:21 < matches> But I'm pretty sure it should be -(83,9)
+23:21 < matches> + even
+23:22 < matches> (The least significant digit is first)
+23:23 < matches> Actually the plot thickens
+23:23 < matches> +45,10 - +128,0 = +18446744073709551533,9
+23:23 < matches> +45,10 - +128 = -83,18446744073709551605
+23:23 <@sulix> What is 0 + -1?
+23:24 < matches> +0 + -1 = -1
+23:24 <@sulix> I am pleased to hear that.
+23:25 < matches> +0 - +1 = -1
+23:25 <@sulix> -1 + 2?
+23:25 < matches> Now you're getting tricky...
+23:25 <@sulix> (As a maths major, I should know this)
+23:25 < matches> -1 - +2 = +1
+23:25 < matches> They're all single digit though
+23:26 <@sulix> Do you mean -1 + +2 = +1?
+23:26 <@sulix> Otherwise I'm slightly concerned.
+23:26 < matches> Uh yeah
+23:26 < matches> Sorry I have to change both the Arbint c(a + b) and the Debug
+23:26 < matches> Before copy pasting
+23:27 < matches> So I guess I should calculate if 45 + 10*2^64 - 128 == 18446744073709551533 + 9*2^64
+23:31 < matches> Expletive grenade
+23:31 < matches> I started Mathematica
+23:31 < matches> fglrx segfaulted
+23:31 < matches> startx
+23:32 < matches> -bash: /usr/bin/startx: No such file or directory
+23:32 < matches> Also now I am getting zenity int3 error messages all over the bottom line of my terminal
+23:33 < matches> Whyyy
+23:33 <@sulix> That's terrible.
+23:33 < matches> Why does this computer persecute me so
+23:34 < matches> I gave it a good life
+23:34 <@sulix> You should ask frames how he runs mathematica on the command line.
+23:34 < matches> I fed it lots of AC power
+23:34 < matches> I replaced that keyboard that I broke
+23:34 < matches> I only spilled coffee on it once
+23:34 < matches> I *tried* to use the drivers that weren't fglrx
+23:35 < matches> I give up, see you tomorrow. Try not to totally rewrite all the Arbint code :P
+23:35 < matches> I think the subtraction might actually work except for the resizing bit
+23:35 <@sulix> Nah, I'm calling it a night here too.
+23:36 <@sulix> I tried replacing the carry bit with the sign bit and all went to hell there, so I'm inclined to agree with you.
+23:37 < matches> Well using my calculator I get something with a 184 ish at the start and too many digits to display
+23:37 < matches> Hence the mathematica disaster
+23:40 < matches> Oh well, I got startx back by installing "xinit" so that's something
+23:41 < matches> I guess apt decided to remove it as part of apt-get autoremove in its infinite wisdom
+23:41 <@sulix> I have decided to never call apt-get autoremove.
+23:42 <@sulix> If at any point there are too many old packages for me to get things done, I'll just reinstall the OS entirely.
+--- Day changed Mon Jul 07 2014
+08:40 < matches> I try to avoid it, but sometimes apt won't let me do anything because it would break some dependency
+08:40 < matches> Although usually even after I run autoremove it complains
+08:41 < matches> Had to use aptitude to fix my "You can'd have kde-runtime!" issues
+08:45 < matches> Anyway wolfram alpha says that example +45,10 - +128 == +18446744073709551533,9
+08:45 < matches> is correct
+08:45 < matches> Checking the algorithm using 8 bits it should work too
+08:46 < matches> So it works as long as you supply the leading zeros...
+08:46 < matches> (On that particular case...)
+08:48 < matches> I hope it's not interpreting Arbint(128L) as Arbint(128u, ...)
+08:48 < matches> Nope sanity seems to prevail there
+08:50 < matches> Ah
+08:50 < matches> Ok
+08:50 < matches> First bug of the day: The subtracted argument needs to be resized
+08:50 < matches> As I am doing (45,10) - (128)
+08:59 < matches> :O
+08:59 < matches> I passed the realops test
+09:00 < matches> And it's also a lot faster now!
+09:00 < matches> Probably because the subtraction bug meant it was spending a long time in division to get the wrong result
+09:01 < matches> Yeah it should have been obvious division can't have been that slow for numbers starting at 64 bits since it does at most as many iterations as the number of bits
+09:01 < matches> Oh well
+09:02 < matches> Should probably not celebrate just yet, now the bezier looks really wierd again
+09:04 < matches> Ok
+09:04 < matches> Actually compiling with Real == Rational<Arbint> might be a better test
+09:04 < matches> But it still passes
+09:05 < matches> In particular it passes with Arbints where int64_t fails horribly
+09:05 < matches> So something is right
+09:05 < matches> Just not all of it...
+09:08 < matches> The GPU rendering seems to workish
+09:09 < matches> We really quickly end up with Arbints of size 30 digits or so :S
+09:09 < matches> Which is big enough to give an overflow in doubles I think
+09:10 < matches> Rational::ToDouble can probably be written as double(integer division) + Rational(remainder, quotient).ToDouble()
+09:10 < matches> (With some things to stop it recursing infinitely)
+09:20 < matches> Or even double(integer division) + double(remainder)/double(quotient)
+09:20 < matches> I don't think any recursion is needed
+09:22 < matches> Oh, there's a "pow" call that is probably broken
+09:24 <@sulix> Yeah, I feel the growing quickly is just a fact of life.
+09:25 <@sulix> Perhaps changing the zooming to avoid coprimes or something.
+09:27 <@sulix> I'm pretty certain that about 3 seconds of zooming is enough to make us need precision and range to match measuring the size of the visible universe in Planck lengths, too.
+09:28 <@sulix> "New biggest Arbint of size 173"
+09:28 <@sulix> and counting...
+09:34 < matches> haha
+09:34 < matches> There are more issues I spotted with add and subtract
+09:34 < matches> Basically you have to add the leading zeros - all of them
+09:35 < matches> My patch only simulates adding one leading zero
+09:35 < matches> Actually maybe I don't need to add them all, just enough until the borrow becomes zero
+09:36 < matches> But whatever
+09:36 < matches> If you're left with a borrow you can't assume that subtracting it from the next digit won't cause a borrow as well
+09:37 < matches> Realops is not a good enough tester
+09:42 <@sulix> Surely if there's borrow bit, subracting it from zero will always give another...?
+09:42 < matches> You only need as many as there are digits in your number
+09:42 < matches> At most
+09:42 < matches> If you still have a borrow at the end
+09:42 < matches> That means the result is negative
+09:43 <@sulix> Yeah, that's what I thought: you need to cap the number of digits you add.
+09:44 < matches> The problem was the algorithm before didn't finish all the borrows
+09:44 < matches> eg: 100 - 2 would have become "-8"
+09:44 < matches> Because 2 only has one digit
+09:44 <@sulix> My powers of mathematics tell me that that is not quite correct.
+09:45 < matches> What, that our algorithm was wrong?
+09:45 <@sulix> No, that 100-2 is not -8
+09:45 < matches> Yes
+09:45 < matches> Yes that's the problem :P
+09:46 < matches> So basically the two numbers need to be the same size but since we have a constant argument I'm sticking in a "borrow_digits" vector if there is a borrow
+09:46 < matches> Addition also needs it for carry
+09:46 < matches> Damn
+09:47 < matches> I'd hoped that might fix the beziers not looking like beziers on the CPU bug
+09:50 <@sulix> I've pushed code to clear the carry/borrow flag before adding/subtracting.
+09:50 <@sulix> I don't think any bug is triggered, but there was the potential for off-by-one errors.
+09:52 < matches> Argh no
+09:52 < matches> Merge
+09:53 < matches> Yeah I'm pushing a fix for bugs that I don't think we've seen (yet)
+09:54 <@sulix> Out of curiosity, I tried removing the Simplify() calls from Rational.
+09:54 <@sulix> Relatedly, I've coined the term "NaN soup"
+09:54 < matches> Oh, clearing the carry/borrow flag might actually fix a bug with the code I just committed
+09:54 < matches> No wait it won't
+09:55 < matches> Eh it's still kind of dangerous to assume it isn't set to start with
+09:55 <@sulix> I think the flag was probably already cleared by the code the compiler generated, but that might not be the case if it's optimized.
+09:55 < matches> One of my early attempts had inline adc instructions in a for loop
+09:56 < matches> Which failed miserably because the for loop was clearing the carry flag :P
+09:57 <@sulix> For some reason "*=" segfaults if optimization is enabled, but I don't know why.
+09:57 < matches> Yuk
+09:58 <@sulix> Somehow we end up with "mul" as a null reference.
+09:58 < matches> Rataional::*= or Arbint?
+09:58 <@sulix> Arbint
+09:58 <@sulix> (So, also rational)
+09:58 <@sulix> Valgrind isn't picking anything up.
+09:58 < matches> null references are the worst
+09:59 < matches> Since I'm pretty sure the point of a reference is that it is never null...
+09:59 < matches> I'm sure this doesn'
+09:59 < matches> t stop people from deliberately creating null references
+09:59 < matches> And also checking that they are not null
+10:02 <@sulix> Tried compiling with clang. Nope.
+10:02 <@sulix> Apparently not even #include <iostream> compiles with clang.
+10:02 < matches> It might be worth implementing a non assembly version of those functions
+10:03 <@sulix> Yeah, that's always a good idea.
+10:03 <@sulix> The assembly ones are more fun, though.
+10:16 < matches> Hmm I should probably head towards the meeting
+16:23 < matches> I have pushed a Gmpint wrapper thing
+16:23 < matches> It mimics Arbint as closely as possible.
+16:24 < matches> Well by that I mean I just wrapped all the calls to mpz_add() etc in operators
+16:24 < matches> It doesn't have all of Arbint's more esoteric functions
+16:24 < matches> If we need bit shifts I think I can add them
+16:27 < matches> So, writing stuff...
+16:28 < matches> Running arbint_vs_gmpint.test in valgrind takes a fair while
+16:28 < matches> We will have the definitive answer of just how terrible our multiplication is compared to the Professional (TM) implementation
+16:29 < matches> I do have some good news, I think our string conversion might be better
+16:29 < matches> Well if you ignore the division
+16:31 < matches> *= is about 11 times slower
+16:32 < matches> Most of that is in std::vector::resize()
+16:32 < matches> But even just mul_digits is greater than the entire Gmpint::operator*=
+16:32 < matches> The plot thickens as I try division
+16:32 < matches> Which is a good opportunity to get food because it will probably take an hour
+16:34 < matches> Note: Don't disable Debug messages if you want time to get food :(
+16:35 < matches> And division is just under 100* slower
+16:36 < matches> No
+16:36 < matches> 1000*
+16:37 < matches> So... if we change Rational<Arbint> to Rational<Gmpint>...
+16:41 < matches> We still get {nan,nan,nan,nan} eventually but it is generally less shoddy
+16:50 <@sulix> Cool: looking at this now.
+16:50 <@sulix> I'm digging through the gmp source code as well.
+16:51 <@sulix> They have a similar "divide by single int64_t" optimization.
+16:51 <@sulix> Also their code is 90% preprocessor macros.
+16:51 < matches> Welp
+16:51 < matches> At least we have a readable Arbint
+16:52 < matches> I don't know what use a readable but slow implementation is
+16:52 <@sulix> Ah: I see what they're doing: this is quite clever, particularly from a pure maths point of view.
+16:52 <@sulix> They've got a natural number implementation, and have then built their integer representation around that.
+16:53 <@sulix> The natural numbler implementation is just a huge set of directories with different assembly implementation.
+16:53 <@sulix> So there's an "x86_64" directory, and in that there's a bunch of assembly + a bunch of directories with optimized versions for different individual processor models.
+16:55 <@sulix> Also their assembly has lots of ASCII art comments.
+16:56 <@sulix> and macros.
+17:07 < matches> Yeah "just use GMP" is probably the answer
+17:08 < matches> Their Makefiles are pretty intimidating
+17:22 < matches> I like that they seem to store the sign as part of the size
+17:22 < matches> If something has a negative size it is negative and has |size| digits
+17:34 < matches> I guess I will try and write some sort of report about how we implemented Arbitrary Integers but they are terrible compared to existing implementations :P
git.ucc.asn.au / ipdf / documents.git / blobdiff
UCC git Repository :: git.ucc.asn.au