22:47 < jtanx> see you tomorrow (probably)
22:48 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 23.0.1/20130814063812]"]
23:28 -!- Callum_ [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 23.0.1/20130814063812]"]
+--- Day changed Fri Sep 13 2013
+08:31 < jtanx> oh yeah, I forgot to restart the bot :P
+08:53 < sam_moore> So, using 2 FILE* causes some problems, I recommend we just use 1
+08:54 < sam_moore> In theory the operating system is supposed to use locks when multiple processes have access to the same file
+08:54 < sam_moore> But...
+08:54 < sam_moore> It probably doesn't work with threads
+08:55 < sam_moore> I'm not sure exactly what goes wrong, but using 1 works and isn't really going to be slower (if the OS uses locks then that probably has a similar overhead to using a mutex)
+08:59 < jtanx> yeah, let's stick to one then
+09:00 < jtanx> there's still a few bugs around I think
+09:00 < sam_moore> Probably
+09:01 < sam_moore> I put valgrind back in, I got rid of a lot last night
+09:01 < jtanx> nice
+09:01 < sam_moore> Although I probably introduced them in the first place
+09:01 < jtanx> what was with that mem leak in fastcgi code?
+09:03 < sam_moore> Oh, that wasn't a memory leak, it was an uninitialised value
+09:04 < sam_moore> I thought I'd be clever and use sizeof() on the FCGIValue array
+09:04 < sam_moore> But I forgot that the actual number of elements is not just sizeof()
+09:04 < sam_moore> It's sizeof(array) / sizeof(element)
+09:04 < jtanx> oh right haha
+09:05 < sam_moore> So the refactored code basically does the same thing
+09:05 < sam_moore> But hopefully it's a bit cleaner
+09:05 < jtanx> yep
+09:05 < jtanx> it's pretty good
+09:06 < jtanx> when you get an invalid id or format, do you want to reject that?
+09:07 < sam_moore> Yeah, I wasn't sure what I should do there, but I think Sensor_Handler is the only place you can really deal with that
+09:07 < jtanx> yeah
+09:07 < sam_moore> I did have it calling FCGI_RejectJSON if the sensor id was invalid
+09:08 < jtanx> the current code on git just calls Log
+09:08 < sam_moore> But I must have done something wrong, because the normal JSON was getting printed just after that
+09:08 < sam_moore> So I removed it
+09:08 < jtanx> oh yeah
+09:08 < jtanx> you have to return after FCGI_RejectJSON
+09:08 < sam_moore> Ah
+09:08 < sam_moore> Durr
+09:08 < jtanx> :P
+09:09 < jtanx> anyways I'll look through and see if I can fix stuff up
+09:09 < jtanx> then I'll try and do that start/stop stuf
+09:10 < sam_moore> Ok
+09:10 < sam_moore> I put in Sensor_Start and Sensor_Stop which might help
+09:10 < sam_moore> I thought you might want to pause data collection without actually "stopping" the experiment?
+09:11 < jtanx> ok
+09:11 < jtanx> that might be a good idea
+09:11 < sam_moore> And also it should be fairly easy to implement a Sensor_Load if we want to be able to load previous experiments
+09:11 < jtanx> yeah
+09:38 < jtanx> those issues with fread
+09:38 < jtanx> it could be because you didn't specify the 'b' flag in fopen
+09:38 < jtanx> ?
+09:39 < jtanx> // Clear the FILE*s
+09:39 < jtanx> df->read_file = NULL;
+09:39 < jtanx> df->write_file = NULL;
+09:39 < jtanx> fclose(df->write_file);
+09:52 < sam_moore> Oh... hurdur
+09:52 < sam_moore> Wait how is it working at all
+09:52 < sam_moore> The write file is mode "w+" which should make it plain text
+09:53 < jtanx> so it's no longer a binary file?
+09:53 < jtanx> I think i may not be experiencing the same issues as you just because I'm running everything on single core processors
+09:53 < sam_moore> Well... it certainly looks like a binary file when you try and cat it
+09:53 < jtanx> ok
+09:53 < jtanx> it's probably a good idea to stick to one file pointer and use mutexes
+09:53 < sam_moore> Yeah
+09:53 < sam_moore> But also make sure that file is binary
+09:53 < jtanx> yeah
+09:54 < sam_moore> It's probably compiler specific what happens when you use fwrite and fread on a plain text file
+09:54 < jtanx> probably
+09:54 < sam_moore> But it looks like a binary file to me, so I'm not sure that's related to the problems with fread
+09:54 < sam_moore> I can investigate this by changing like 4 characters...
+09:55 < jtanx> one thing I did notice on linux
+09:55 < jtanx> if it wasn't a binary file you'd have to use fflush to explicitly write out text and have it appear when you used cat
+09:56 < jtanx> but overall I think it's not due to it being a binary file or not
+09:56 < sam_moore> Yep
+09:56 < sam_moore> I just put the second FILE* for read_file back in
+09:56 < sam_moore> Made them both binary
+09:56 < sam_moore> Still get the "Transport endpoint is not connected" errors
+09:57 < sam_moore> So there should be a "b" in the file mode, but that's unrelated to having multiple FILE*
+09:57 < jtanx> yep then I think that's due to you running it on a multicore processor
+09:57 < jtanx> maybe
+09:57 < sam_moore> I think the OS locks around file access are based on PID
+09:58 < jtanx> on linux all file locks are advisory only
+09:58 < sam_moore> Can
+09:58 < sam_moore> Anyway, it's clearly safest to just have one FILE*
+09:58 < jtanx> yeah
+09:58 < jtanx> I'm changing it now
+10:09 < jtanx> oh ffs that's the second time it's crashed simply because I didn't do make clean before make
+10:16 < sam_moore> haha
+10:17 < sam_moore> Modify the make file to always run make clean?
+10:28 < jtanx> haha that's one option
+10:28 < jtanx> it's because make doesn't consider changes to the header files
+10:28 < jtanx> there's a way to make it detect those, but *effort*
+10:40 < sam_moore> Regarding Actuators, I think if we design for a thread to control (one or more) actuators that will make it easy to add more advanced control later
+10:42 < sam_moore> eg: The Actuator_Handler just sets some kind of formula or says "Every time sensor 0 is stable for Xs (within a range) change actuator by Y" and the thread watches the sensor data and changes the actuator when necessary
+10:43 < sam_moore> You could also add feedback that way too
+10:44 < sam_moore> "Set this actuator so that sensor 0 has value X"
+10:46 < jtanx> hmm
+10:47 < jtanx> about the sensor stuff
+10:47 < jtanx> there were a few things I noticed
+10:48 < jtanx> the time wrapping
+10:48 < jtanx> could still give a negative start_time or end_time
+10:48 < jtanx> if the user specified a very negative start_time/end_time
+10:49 < jtanx> (eg specifies a negative time greater than the total running time)
+10:49 < sam_moore> Oh yeah
+10:49 < jtanx> so in data_printbytimes
+10:49 < jtanx> I just clamped it to zero
+10:49 < jtanx> instead of asserting
+10:49 < sam_moore> Fair enough
+10:49 < jtanx> strangely this code seems to work pretty good:
+10:49 < jtanx> void Data_PrintByTimes(DataFile * df, double start_time, double end_time, DataFormat format)
+10:49 < jtanx> {
+10:49 < jtanx> assert(df != NULL);
+10:49 < jtanx> //Clamp boundaries
+10:49 < jtanx> if (start_time < 0)
+10:49 < jtanx> start_time = 0;
+10:50 < jtanx> if (end_time < 0)
+10:50 < jtanx> end_time = 0;
+10:50 < jtanx> int start_index = 0, end_index = 0;
+10:50 < jtanx> if (start_time < end_time)
+10:50 < jtanx> {
+10:50 < jtanx> start_index = Data_FindByTime(df, start_time, NULL);
+10:50 < jtanx> end_index = Data_FindByTime(df, end_time, NULL);
+10:50 < jtanx> }
+10:50 < jtanx> Data_PrintByIndexes(df, start_index, end_index, format);
+10:50 < jtanx> }
+10:50 < jtanx> oh but that was the other ting
+10:50 < jtanx> it works better if you have exclusive end index/time
+10:50 < sam_moore> Ok
+10:50 < sam_moore> I have to go now
+10:50 < jtanx> ok
+10:50 < sam_moore> See you this afternoon if you're still there
+10:50 < jtanx> yep see you then
+10:51 < jtanx> probably
+13:44 < callum> should i just add the sensor threshold values to the sensor struct?
+13:47 < jtanx> probably not
+13:47 < jtanx> I think you have to create a new struct
+13:48 < jtanx> are you at uni?
+13:49 < callum> yea. why is a new struct necessary?
+13:49 < jtanx> come to g19
+13:49 < jtanx> the bbb is setup too
+16:17 < sam_moore> Are you guys still in G19?
+16:17 < sam_moore> On my way now
+18:42 < jtanx> lol you need to use a transistor to turn on an led from the gpio pins
+19:56 < jtanx> ok I think I'll just escape all input parameters from within the request loop
+19:56 < jtanx> then at least you're guaranteed that user input won't stuff up anything
+19:56 < jtanx> (as in making it generate invalid JSON)
+20:53 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 23.0.1/20130814063812]"]
+--- Day changed Sat Sep 14 2013
+11:32 < sam_moore> blergh, too many different versions of OpenCV
+11:32 < sam_moore> Also too many different data types
+11:33 < sam_moore> "Getting a digital image into numbers can be very handy for your image processing. In this post i will brief just that."
+11:34 < sam_moore> Thank god, finally a tutorial that makes sense
+11:34 -!- Irssi: #mctxuwa_softdev: Total of 2 nicks [0 ops, 0 halfops, 0 voices, 2 normal]
+11:34 < sam_moore> ... no one else is in the channel
+11:34 < sam_moore> :S
+12:36 < jtanx> on start/stop/pausing an experiment
+12:36 < jtanx> currently everything is relative to g_options.start_time
+12:42 < sam_moore> Yeah
+12:43 < sam_moore> Perhaps make an ExperimentTime function and that leaves us free to change what the time stamps are relative to?
+12:43 < sam_moore> Looking into this interferometer stuff...
+12:43 < jtanx> ok, maybe as part of the Control_ module then
+12:43 < sam_moore> Interesting problem, I think I've worked out an algorithm that isn't too terrible
+12:44 < jtanx> yeah?
+12:44 < jtanx> how does it work
+12:45 < sam_moore> If you have a sinusoidal wave, find the x intercepts, and that lets you calculate the phase and frequency
+12:45 < sam_moore> That's the simple version
+12:46 < jtanx> haha alright
+12:46 < sam_moore> I have 3 pages of notes that I can show Adrian/Adam if I can't get it working
+12:46 < jtanx> wow
+12:46 < jtanx> nice work
+12:47 < sam_moore> Fortran would be good for the array operations
+12:47 < sam_moore> If fortran were able to do anything *other* than array operations it would be a nice language...
+12:47 < jtanx> have you ever used Fortan before?
+12:48 < sam_moore> Yes
+12:48 < sam_moore> I'm going to keep it in C though
+12:48 < sam_moore> It took long enough to get OpenCV working
+12:48 < sam_moore> I'm sure there's some kind of convoluted Fortran library
+12:49 < sam_moore> But a seperate fortran program won't talk to our server
+12:49 < jtanx> can you compile the code as a library with C interop
+12:49 < sam_moore> Erm..
+12:49 < sam_moore> Probably more effort than just doing it in C
+12:49 < jtanx> probably
+12:51 < jtanx> I think a separate mutex is needed around the *_Stop() functions
+12:51 < jtanx> because if a client sends in a request
+12:51 < sam_moore> Oh, good point
+12:52 < jtanx> although that's only going to be a problem if our own software calls the stop function because of the sanity checks
+12:52 < sam_moore> If there are 2 requests?
+12:52 < sam_moore> No wait, the FCGI stuff is serial
+12:52 < jtanx> the request loop is single threaded
+12:52 < sam_moore> It has to finish processing one request to respond to the next
+12:52 < sam_moore> That's fine
+12:56 < sam_moore> Put a mutex in the Sensor struct, use the same one that's already in the Actuator struct, put critical sections around setting the "s->record_data" or "a->activated" variables (or whatever you choose to use as a "is thing running" flag)
+12:56 < sam_moore> And check the flag hasn't already been set before doing anything
+12:57 < sam_moore> Similar to how the Thread_Running() and Thread_QuitProgram() functions used to work
+12:57 < sam_moore> .... Before I deleted them
+12:57 < sam_moore> Any thread could call Thread_QuitProgram, and repeated calls would have no effect
+13:01 < jtanx> ok I'll try that
+13:51 < jtanx> actually, what about this:
+13:51 < jtanx> in control.c, it maintains its own mutex
+13:51 < jtanx> so
+13:52 < jtanx> and it's the only place where Sensor_Start/Stop Actuator_Start/Stop is called
+13:53 < jtanx> You then have Control_Lock/Unlock that's placed around the responses in the handler functions
+13:53 < jtanx> those functions internally lock/unlock the mutex held in control.c
+13:53 < jtanx> ah stuff it
+13:53 < jtanx> I'll commit it to my git repo
+13:54 < jtanx> I still don't know how the pause functionality should be done though
+13:54 < jtanx> especially with the time discontinuities
+14:01 < jtanx> oh yeah, just a reminder that you have to be careful using FCGI_LONG_T vs FCGI_INT_T
+14:02 < jtanx> if your variable is of type int but you give FCGI_LONG_T, and where sizeof(int) != sizeof(long) then there could be a buffer overflow
+14:13 < sam_moore> Right
+14:14 < sam_moore> If you want to commit to the main repo you can use a new branch
+14:14 < sam_moore> Having a single mutex in control.c instead of one per sensor/actuator is probably better
+14:15 < sam_moore> With the pause functionality... I think if you just don't call Data_Close() (and don't call Data_Open on resuming) but otherwise stop the threads running Sensor_Loop in the same way
+14:15 < sam_moore> That should do most of it
+14:16 < sam_moore> Don't reset the experiment start time
+14:17 < sam_moore> Perhaps the experiment start time needs to be stored somewhere though
+14:18 < sam_moore> Anyway, I'll let you solve the problem :P
+14:48 < jtanx> ._.
+15:45 < jtanx> well I think I got something semi-working
+15:45 < jtanx> it involved adding on Sensor_PauseAll/ResumeAll + Acuator_PauseAll/ResmueAll (plus the individual Pause/Resume) functions
+15:46 < sam_moore> Cool
+15:46 < sam_moore> I think I'm starting to get my head around OpenCV
+15:46 < sam_moore> The documentation for it is a bit crap
+15:46 < jtanx> oh don't get me started on opencv documentation
+15:47 < jtanx> yesterday it took me and callum around 30 mins to figure out that CvMat structure
+15:47 < sam_moore> Haha
+15:47 < sam_moore> It probably took me a bit longer :S
+15:47 < jtanx> the problem is that most of it's geared towards c++ usage
+15:49 < jtanx> back on that control stuff, basically those Sensor/Actuator_[Start|Stop|Pause|Resume] functions can only be called via the Control_[Start|Stop|Pause|Resume] functions
+15:50 < jtanx> partly because of threading issues but also because it's the only place where the current state is currently kept track of
+15:51 < jtanx> (e.g if you call Sensor_Start twice then...
+15:53 < sam_moore> Seems sensible
+15:53 < jtanx> anyway, I'm taking a break from this stuff for a while
+15:53 < sam_moore> Yeah, fair enough
+15:54 < jtanx> If you want to see what I've done so far, it's on my git repo (haven't added comments yet)
+15:54 < sam_moore> Cool
+16:50 < sam_moore> The IplImage vs CvMat conversion is dumb
+16:50 < sam_moore> In fact I don't think you even need to do it?
+16:51 < sam_moore> Well at least to display an image you can just pass a CvMat
+16:51 < sam_moore> Maybe you still need the IplImage to capture from a camera
+16:51 < sam_moore> I worked out how to convert from IplImage to CvMat anyway
+16:53 < sam_moore> Other than that, OpenCV doesn't actually seem horrible to use
+16:53 < sam_moore> Just... contradictory documentation
+16:55 < sam_moore> Anyway... I've created a moving sinusoidal image!
+16:56 < sam_moore> Now to work out why the algorithm keeps returning -nan :S
+16:56 < sam_moore> Also for some reason the image is blue instead of red
+16:56 < sam_moore> But whatever
+16:57 < jtanx> :S
+16:57 < jtanx> BGR vs RGB?
+17:02 < sam_moore> Looks like it
+17:03 < sam_moore> Doing this with an actual camera is going to suck
+17:04 < sam_moore> See, I've worked out an algorithm to cope with changing background light conditions
+17:04 < sam_moore> Because differentiating a sinusoid gives you another sinusoid with the same phase (offset by pi/2)
+17:05 < sam_moore> Buut... differentiating with finite differences adds more noise
+17:05 < sam_moore> Or rather it makes the noise more pronounced
+17:07 < sam_moore> Hopefully sensors is considering this
+17:07 < sam_moore> If they want to shoot a laser into a camera, they should really do it in a way that keeps out background light
+17:07 < sam_moore> Buuut
+17:07 < sam_moore> I think they mentioned using one camera to do both the interferometer and look at the can
+17:07 < sam_moore> :S
+17:08 < sam_moore> Oh well, the optics is their problem
+17:09 < sam_moore> I suppose I will prepare a report or something about the algorithm and what conditions the image needs to satisfy
+17:12 < jtanx> um
+17:12 < jtanx> there's going to be two cameras
+17:12 < jtanx> because it was too much hassle moving it
+17:12 < sam_moore> Yes... but one was meant to be looking at each can?
+17:13 < jtanx> oh
+17:13 < jtanx> hmm
+17:13 < sam_moore> We should ask them
+17:13 < jtanx> I just thought that one would be for the interferometer (no need to look at can) and the other would be for the can to be blown up
+17:13 < sam_moore> Probably
+17:15 < jtanx> but I was thinking for the one to be blown up
+17:16 < jtanx> you could possibly just stream it using something like ffserver
+17:16 < jtanx> instead of passing it through opencv
+17:16 < sam_moore> Yeah, that seems like it's probably easier
+17:23 -!- jtanx [
[email protected]] has quit ["my antivirus is pestering me to restart"]
+18:19 < sam_moore> So... when you take an image with a camera the pixels are stored as rgb with 0->255 values
+18:20 < sam_moore> You can convert the IplImage to a CvMat and it keeps the values ordered as rgb with 0->255 values
+18:20 < sam_moore> And then...
+18:20 < sam_moore> If you try and display it...
+18:20 < sam_moore> The display function treats it as having values in bgr order with 0->1 values
+18:20 < sam_moore> -_-
+18:21 < sam_moore> (So unless you manually adjust all the values you'll just get a white picture in the display)
+18:54 < jtanx> hahaha
+18:54 < jtanx> what
+18:56 < jtanx> for cvShowImage:
+18:56 < jtanx> If the image is 8-bit unsigned, it is displayed as is.
+18:56 < jtanx> If the image is 16-bit unsigned or 32-bit integer, the pixels are divided by 256. That is, the value range [0,255*256] is mapped to [0,255].
+18:56 < jtanx> If the image is 32-bit floating-point, the pixel values are multiplied by 255. That is, the value range [0,1] is mapped to [0,255].
+18:57 < jtanx> so it's treating it as 32-bit floating point?
+20:41 < sam_moore> I think it's because of how I converted the IplImage to a CvMat
+20:42 < sam_moore> My algorithm is almost working
+20:43 < sam_moore> The fact that angles wrap around every 2*pi is giving me a headache though
+20:43 < sam_moore> This is pretty fun
+20:45 < sam_moore> Hopefully I can do Computer Vision as an optional, I think its in the list
+20:45 < sam_moore> Maybe I shouldn't make all my optionals CS units, but meh
+21:20 < jtanx> :P
+21:20 < jtanx> the computer vision unit sounds interesting, but a lot of maths/theory involved
+22:02 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 23.0.1/20130814063812]"]
git.ucc.asn.au / matches / MCTX3420.git / commitdiff
UCC git Repository :: git.ucc.asn.au