+--- Day changed Thu Sep 12 2013
+09:36 < sam_moore> I think I have some time after the tutorial today to work on stuff
+09:36 < sam_moore> I take it we have the bbb now?
+09:37 < sam_moore> I'll bring that crappy webcam
+09:37 < sam_moore> Maybe look into sending images as part of the API, see if it's feasable to do it that way
+09:38 < sam_moore> I think we can introduce start/stop control fairly easily
+09:40 < sam_moore> "Mush Mush"
+09:40 < sam_moore> Heh
+09:41 < sam_moore> To reduce the load on the client, we might be able to send averaged data points instead of every data point
+09:41 < sam_moore> You specify "averages per point"
+09:42 < sam_moore> So each [time, value] pair is actually an average of X recordings
+09:42 < sam_moore> In fact it's better to send [mean_time, mean_value, std_dev]
+09:42 < sam_moore> The std_dev gives you an idea of noise and/or if the value is changing a lot
+09:43 < sam_moore> And of course you can always specify a single average and it becomes equivelant to what we currently do
+09:50 < jtanx> yeah
+09:50 < jtanx> I'm free after the tute
+09:50 < jtanx> I'm bringing in a router to make it easier
+09:50 < jtanx> to work on the bbb together
+09:51 < jtanx> yesterday it was so painful trying to get shit working
+09:51 < jtanx> dhcp issues, dns issues, internet connection sharing issues...
+09:51 < jtanx> (mostly the fault of my own computer though)
+09:51 < jtanx> btw I'm working on adding an extra function to FCGI to makeit easier to parse parameters
+09:51 < jtanx> (hoepfully)
+09:55 < jtanx> gtg see you at the tute
+09:55 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 23.0.1/20130814063812]"]
+18:26 < Callum> would be easier to talk here...if everyone was on
+18:41 < jtanx> did you bring the bbb home?
+18:43 < sam_moore> Yep
+18:44 < sam_moore> I'll put it back tomorrow morning
+18:49 < jtanx> ok
+19:47 < jtanx> I'm so tempted to use bitflags for that request function
+19:52 < jtanx> one thing that I need to do is to sanitise json string fields
+20:18 < sam_moore> Sanitise? As in "make sane" or "make clean"?
+20:18 < sam_moore> Um... if you think bitflags are necessary, I fully support you :P
+20:19 < jtanx> as in
+20:19 < jtanx> If I allow FCGI_JSONPair to also be passed a format string
+20:19 < jtanx> then having stuff like \ or " or newline characters for eg in the string
+20:20 < jtanx> is not a good idea
+20:20 < sam_moore> Hmm
+20:20 < jtanx> it's sorta like this:
+20:20 < sam_moore> Well, writing the function to print out data (either as TSV or JSON)
+20:20 < sam_moore> I've only been using FCGI_PrintRaw
+20:20 < jtanx> yeah
+20:20 < jtanx> but right now, you're logging messages in Server_handler
+20:20 < jtanx> about invalid values
+20:21 < sam_moore> Yes, I'm not sure how to deal with that
+20:21 < jtanx> Sensor_handler*
+20:21 < jtanx> so
+20:21 < jtanx> eg Log(LOGERR, "Require \"all\" or an integer value: %s = %s", key, value);
+20:21 < sam_moore> Yes
+20:21 < jtanx> if I wanted to send that back to the client in FCGI_RejectJSON
+20:21 < jtanx> there's the potential (...) that the user could hvae included a " in the value
+20:22 < jtanx> so if you return it as part of your json response, then suddenly it's broken
+20:22 < sam_moore> Oh, that sucks
+20:23 < sam_moore> Perhaps treat Log messages seperately from JSON responses?
+20:23 < jtanx> eg: "description":" required an id, got 'blah-stray"'"
+20:23 < jtanx> yeah
+20:24 < jtanx> What I could do is just replace any special characters with space
+20:24 < jtanx> and return that in description
+20:24 < jtanx> but keep the actual value in log
+20:24 < sam_moore> Maybe
+20:24 < sam_moore> You could also call FCGI_RejectJSON as soon as you get a special character
+20:25 < sam_moore> Since we shouldn't have any key/values that use them
+20:25 < sam_moore> (I think :P)
+20:25 < jtanx> lol
+20:25 < sam_moore> Whatever seems best
+20:25 < jtanx> let me think about it some more
+20:25 < sam_moore> Yeah, fair enough
+20:25 < sam_moore> Um, another thing (sorry) is error messages that you might get independent of any request
+20:26 < sam_moore> Like you suddenly have an I/O error on a file
+20:26 < sam_moore> Currently we handle that with Fatal
+20:26 < sam_moore> Which it would be convenient to keep; but instead of terminating the whole program, we need to notify the client somehow
+20:26 < jtanx> that won't happen until the next request
+20:27 < jtanx> what you could do is keep it exiting
+20:27 < jtanx> when that happens, nginx sends 502 bad gateway
+20:27 < jtanx> then if that's detected, send a link to the log file
+20:27 < sam_moore> Yeah, that seems good
+20:27 < jtanx> and a method to restart the software
+20:28 < sam_moore> Do we want to notify the client of non-fatal errors or warnings though?
+20:28 < sam_moore> Hmm
+20:29 < jtanx> hmm
+20:29 < jtanx> probably a good idea
+20:29 < jtanx> but how
+20:29 < sam_moore> The client could periodically request the log file and deal with messages?
+20:30 < jtanx> It could just display it I guess
+20:30 < jtanx> onus on the user to take action if required
+20:30 < sam_moore> Sure, an alert might be nice though
+20:30 < sam_moore> Perhaps have an extra log file that just has the most recent message in it (with a timestamp)
+20:31 < jtanx> you could perform a regex on the text and highlight lines with warning or something
+20:31 < sam_moore> The client just periodically looks at that and if it's a new message can alert
+20:31 < sam_moore> Yeah that would work, as long as the file doesn't get too big
+20:31 < sam_moore> It probably won't
+20:31 < jtanx> well you could autoscroll it anyway
+20:32 < jtanx> but ok maybe that's getting a bit ahead of ourselves
+20:32 < sam_moore> Yeah, sure
+20:32 < sam_moore> I'll get back to rewriting 90% of the sensor code :P
+20:32 < jtanx> haha
+20:32 < jtanx> currently, I've got
+20:32 < jtanx> bool FCGI_ParseRequest(FCGIContext *context, char *params, FCGIValue values[], size_t count);
+20:32 < jtanx> (any better suggestion for FCGIValue?)
+20:32 < jtanx> (the name that is)
+20:33 < jtanx> with
+20:33 < sam_moore> That's probably fine
+20:33 < jtanx> typedef struct FCGIValue {
+20:33 < jtanx> const char *key;
+20:33 < jtanx> void *value;
+20:33 < jtanx> unsigned flags;
+20:33 < jtanx> } FCGIValue;
+20:33 < jtanx> you then do
+20:33 < jtanx> something like:
+20:33 < jtanx> FCGIValue values[2] = {{"sensors", &ident_sensors, FCGI_REQUIRED(FCGI_BOOL_T)},
+20:33 < jtanx> {"actuators", &ident_actuators, FCGI_BOOL_T}};
+20:33 < sam_moore> Looks good
+20:33 < jtanx> although I'm open to suggestions on those type names too
+20:34 < jtanx> if you really have to
+20:34 < jtanx> you can do something like FCGI_RECEIVED(values[0].flags)
+20:34 < jtanx> to check if you received the argument or not
+20:34 < sam_moore> If it's not too hard to implement, then OK
+20:35 < jtanx> ?
+20:35 < sam_moore> That looks reasonably nice
+20:35 < jtanx> the #defines look less so:
+20:35 < jtanx> #define FCGI_PARAM_REQUIRED (1 << 0)
+20:35 < jtanx> #define FCGI_PARAM_RECEIVED (1 << 1)
+20:35 < jtanx> #define FCGI_BOOL_T (1 << 2)
+20:35 < jtanx> #define FCGI_LONG_T (1 << 3)
+20:35 < jtanx> #define FCGI_DOUBLE_T (1 << 4)
+20:35 < jtanx> #define FCGI_STRING_T (1 << 5)
+20:35 < jtanx> #define FCGI_REQUIRED(x) ((x) | FCGI_PARAM_REQUIRED)
+20:35 < jtanx> #define FCGI_IS_REQUIRED(x) ((x) & FCGI_PARAM_REQUIRED)
+20:35 < jtanx> #define FCGI_RECEIVED(x) ((x) & FCGI_PARAM_RECEIVED)
+20:35 < jtanx> #define FCGI_TYPE(x) ((x) & ~(FCGI_PARAM_REQUIRED | FCGI_PARAM_RECEIVED))
+20:35 < sam_moore> In fact it's similar to how you use select(2)
+20:35 < sam_moore> Wow, macros
+20:35 < jtanx> :P
+20:35 < jtanx> makes life easier
+20:36 < sam_moore> Alright, this is C
+20:36 < sam_moore> Yeah, FCGI_RECEIVED(values[i].flags) looks nice (from the point of view of actually using it anyway)
+20:37 < jtanx> yeah it's ok
+20:37 < jtanx> the only stickler is that hardcoded index
+20:37 < jtanx> can't really do much about that though
+20:37 < sam_moore> I can put an enum in the function
+20:38 < jtanx> if you want; it might be more work to maintain the enum than anything though
+20:39 < jtanx> but yeah, essentially if you require a parameter, you surround the type in FCGI_REQUIRED()
+20:39 < jtanx> hopefully that part was clear
+20:39 < sam_moore> Yep, I'm happy with that
+20:39 < jtanx> cool
+20:43 < jtanx> for the bool value, what behaviour do you think is better
+20:43 < jtanx> always set the value to true, or invert the current value
+20:44 < jtanx> right now it's set to always be true
+21:06 < sam_moore> Um... always set to true?
+21:09 < jtanx> yeah
+21:29 < jtanx> oh
+21:30 < jtanx> well I didn't need to escape after all
+21:30 < jtanx> If you do something like this: http://mctx.us.to:8080/api?afa%22%22%22%22
+21:30 < jtanx> it gets url encoded
+21:30 < jtanx> http://mctx.us.to:8080/api?afa""""
+21:30 < jtanx> I wonder if that's guaranteed
+21:32 < jtanx> well, apparently it's not because IE doesn't escape it for you
+21:32 < jtanx> so I did need to escape
+21:33 < sam_moore> There's no escape
+21:46 < jtanx> wait wat
+21:59 < sam_moore> (It's a pun)
+22:00 < sam_moore> So I've made a DataFile structure to isolate it a bit more from the higher level Sensor related stuff
+22:00 < sam_moore> Hopefully things make more sense
+22:00 < sam_moore> I put in 2 seperate FILE* as well
+22:00 < sam_moore> And got rid of the confusing "fill a buffer then dump the buffer to a file" loop
+22:01 < sam_moore> Now it just writes each point to a file
+22:01 < sam_moore> Hopefully it's thread safe enough
+22:01 < sam_moore> Now... I just need to simplify the Sensor_Handler function...
+22:02 < jtanx> > (It's a pun)
+22:02 * jtanx facepalm
+22:03 < jtanx> hoho
+22:03 < jtanx> ok
+22:03 < jtanx> I'll submit what I've done so far to git
+22:08 < jtanx> callum
+22:08 < jtanx> that code you submitted
+22:08 < jtanx> I don't think it's right
+22:08 -!- Irssi: #mctxuwa_softdev: Total of 3 nicks [0 ops, 0 halfops, 0 voices, 3 normal]
+22:09 < jtanx> the stuff with g_sensor_names, that's the human readable name of each sensor
+22:10 < sam_moore> I've changed the name and how the sanity check gets called, but I've left the body unimplemented for now
+22:12 < jtanx> it's with the pull request he submitted ~30mins ago
+22:12 < sam_moore> Oh, ok
+22:13 < sam_moore> Good thing I made a new branch for my changes
+22:14 < jtanx> the pull request isn't merged yet
+22:15 < sam_moore> Yeah, I don't think we should merge that, sorry Callum
+22:17 < jtanx> ok just submitted the pull request
+22:17 < sam_moore> I never put documentation on g_sensor_names I guess, sorry
+22:17 < jtanx> I think I did though
+22:17 < jtanx> but in the header file
+22:18 < jtanx> which is annoying :/
+22:18 < jtanx> (as in there's more than one spot to put documentation)
+22:19 < sam_moore> Yeah... does one overwrite another?
+22:19 < sam_moore> I'll just copy paste the same thing into the .c file
+22:19 < jtanx> No idea how doxygen handles that
+22:20 < sam_moore> I wonder how much fun I'll have merging these branches...
+22:20 < jtanx> hehe
+22:20 < jtanx> I'm not really sure if FCGI_ParseRequest will be that helpful or not
+22:20 < jtanx> if you find it doesn't really help then we can scrap it
+22:21 < sam_moore> Well it makes sense to seperate the code for getting key,value pairs from the code that does stuff with them
+22:23 < jtanx> I guess
+22:23 < sam_moore> I've basically moved a lot of functionality from sensor.c into a new file data.c under functions with the prefix "Data_"
+22:24 < jtanx> ok
+22:24 < sam_moore> data.c doesn't care anything about what the data is or where it came from, it just wraps around saving/reading/selecting ranges of DataPoints
+22:25 < jtanx> sounds like a good idea
+22:25 < jtanx> it might be of use in actuator code I guess
+22:25 < sam_moore> Yeah, there's a DataFile structure that basically wraps around the binary file
+22:25 < sam_moore> Yes, I was thinking that too
+22:25 < sam_moore> Because we'll probably want to store DataPoints when an actuator gets changed
+22:26 < jtanx> yeah true
+22:27 < sam_moore> If the idea about using 2 FILE* is right... I don't think we really need any mutexes?
+22:28 < sam_moore> There is an index that gets incremented, but since only one thread can write to it, the worst that can happen is you return one (maybe two at the extreme) less point than actually exists
+22:29 < sam_moore> If you had 2 threads incrementing the index you'd need a mutex though
+22:29 < jtanx> the problem is that setting a value is not guaranteed to be atomic
+22:29 < jtanx> i think
+22:29 < sam_moore> Yes, sure
+22:29 < jtanx> so while setting and you read it it could be indeterminate
+22:29 < sam_moore> Mmm
+22:30 < jtanx> I think you need mutexes around reading/writing the points_written(?)
+22:30 < jtanx> index
+22:30 < jtanx> but that's it
+22:31 < jtanx> http://stackoverflow.com/questions/54188/are-c-reads-and-writes-of-an-int-atomic
+22:31 < sam_moore> I'm not sure you do if you are writing in one thread and reading in another
+22:31 < sam_moore> Ah, stackoverflow, OK
+22:32 < jtanx> it's actually very architecture specific
+22:32 < sam_moore> Yeah "Yes, no, hmmm, well it depends" is probably right
+22:32 < jtanx> haha
+22:32 < sam_moore> Well... there are no mutexes at the moment, but maybe I should get around to putting them in
+22:32 < sam_moore> I think I had one thread writing and another reading in my parallel programming assignment and got full marks :S
+22:33 < sam_moore> Threads are strange and magical
+22:33 < jtanx> hey well it probably works 99% of the time
+22:34 < sam_moore> Ah, it probably works if the size of the integer is smaller than the bus
+22:34 < sam_moore> smaller or equal
+22:34 < jtanx> too bad linux doesn't have something like InterlockedExchange
+22:35 < sam_moore> I'll add mutexes in later then
+22:35 < jtanx> One thing I liked about C++ was auto locks
+22:36 < jtanx> (you could call some lock function and once that scope ended the lock ended too)
+22:36 < sam_moore> That's nice
+22:40 < jtanx> Ok, that's all I'm probably going to get done today
+22:41 < sam_moore> Yeah, thanks
+22:42 < jtanx> what's with the meeting tomorrow though
+22:43 < sam_moore> I'm busy from 2:00pm until at least 4:00pm (maybe later)
+22:43 < sam_moore> But I think I should be preparing for something before 2:00pm
+22:44 < jtanx> ok, I'll probably come in around 1.30pm so I can work with Callum, and I'll stay on until whatever time
+22:44 < sam_moore> Alright
+22:45 < sam_moore> Check github in the morning, I'll push all my changes
+22:45 < jtanx> ok I'll do that
+22:45 < sam_moore> Hopefully they make sense, otherwise you can change everything back :P
+22:45 < sam_moore> The wonders of git
+22:45 < jtanx> haha I'm sure it'll make sense
+22:46 < jtanx> after staring for a while
+22:46 < sam_moore> Yeah it should be a bit simpler
+22:46 < jtanx> thanks
+22:47 < jtanx> see you tomorrow (probably)
+22:48 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 23.0.1/20130814063812]"]
+23:28 -!- Callum_ [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 23.0.1/20130814063812]"]
+--- Day changed Fri Sep 13 2013
+08:31 < jtanx> oh yeah, I forgot to restart the bot :P
+08:53 < sam_moore> So, using 2 FILE* causes some problems, I recommend we just use 1
+08:54 < sam_moore> In theory the operating system is supposed to use locks when multiple processes have access to the same file
+08:54 < sam_moore> But...
+08:54 < sam_moore> It probably doesn't work with threads
+08:55 < sam_moore> I'm not sure exactly what goes wrong, but using 1 works and isn't really going to be slower (if the OS uses locks then that probably has a similar overhead to using a mutex)
+08:59 < jtanx> yeah, let's stick to one then
+09:00 < jtanx> there's still a few bugs around I think
+09:00 < sam_moore> Probably
+09:01 < sam_moore> I put valgrind back in, I got rid of a lot last night
+09:01 < jtanx> nice
+09:01 < sam_moore> Although I probably introduced them in the first place
+09:01 < jtanx> what was with that mem leak in fastcgi code?
+09:03 < sam_moore> Oh, that wasn't a memory leak, it was an uninitialised value
+09:04 < sam_moore> I thought I'd be clever and use sizeof() on the FCGIValue array
+09:04 < sam_moore> But I forgot that the actual number of elements is not just sizeof()
+09:04 < sam_moore> It's sizeof(array) / sizeof(element)
+09:04 < jtanx> oh right haha
+09:05 < sam_moore> So the refactored code basically does the same thing
+09:05 < sam_moore> But hopefully it's a bit cleaner
+09:05 < jtanx> yep
+09:05 < jtanx> it's pretty good
+09:06 < jtanx> when you get an invalid id or format, do you want to reject that?
+09:07 < sam_moore> Yeah, I wasn't sure what I should do there, but I think Sensor_Handler is the only place you can really deal with that
+09:07 < jtanx> yeah
+09:07 < sam_moore> I did have it calling FCGI_RejectJSON if the sensor id was invalid
+09:08 < jtanx> the current code on git just calls Log
+09:08 < sam_moore> But I must have done something wrong, because the normal JSON was getting printed just after that
+09:08 < sam_moore> So I removed it
+09:08 < jtanx> oh yeah
+09:08 < jtanx> you have to return after FCGI_RejectJSON
+09:08 < sam_moore> Ah
+09:08 < sam_moore> Durr
+09:08 < jtanx> :P
+09:09 < jtanx> anyways I'll look through and see if I can fix stuff up
+09:09 < jtanx> then I'll try and do that start/stop stuf
+09:10 < sam_moore> Ok
+09:10 < sam_moore> I put in Sensor_Start and Sensor_Stop which might help
+09:10 < sam_moore> I thought you might want to pause data collection without actually "stopping" the experiment?
+09:11 < jtanx> ok
+09:11 < jtanx> that might be a good idea
+09:11 < sam_moore> And also it should be fairly easy to implement a Sensor_Load if we want to be able to load previous experiments
+09:11 < jtanx> yeah
+09:38 < jtanx> those issues with fread
+09:38 < jtanx> it could be because you didn't specify the 'b' flag in fopen
+09:38 < jtanx> ?
+09:39 < jtanx> // Clear the FILE*s
+09:39 < jtanx> df->read_file = NULL;
+09:39 < jtanx> df->write_file = NULL;
+09:39 < jtanx> fclose(df->write_file);
+09:52 < sam_moore> Oh... hurdur
+09:52 < sam_moore> Wait how is it working at all
+09:52 < sam_moore> The write file is mode "w+" which should make it plain text
+09:53 < jtanx> so it's no longer a binary file?
+09:53 < jtanx> I think i may not be experiencing the same issues as you just because I'm running everything on single core processors
+09:53 < sam_moore> Well... it certainly looks like a binary file when you try and cat it
+09:53 < jtanx> ok
+09:53 < jtanx> it's probably a good idea to stick to one file pointer and use mutexes
+09:53 < sam_moore> Yeah
+09:53 < sam_moore> But also make sure that file is binary
+09:53 < jtanx> yeah
+09:54 < sam_moore> It's probably compiler specific what happens when you use fwrite and fread on a plain text file
+09:54 < jtanx> probably
+09:54 < sam_moore> But it looks like a binary file to me, so I'm not sure that's related to the problems with fread
+09:54 < sam_moore> I can investigate this by changing like 4 characters...
+09:55 < jtanx> one thing I did notice on linux
+09:55 < jtanx> if it wasn't a binary file you'd have to use fflush to explicitly write out text and have it appear when you used cat
+09:56 < jtanx> but overall I think it's not due to it being a binary file or not
+09:56 < sam_moore> Yep
+09:56 < sam_moore> I just put the second FILE* for read_file back in
+09:56 < sam_moore> Made them both binary
+09:56 < sam_moore> Still get the "Transport endpoint is not connected" errors
+09:57 < sam_moore> So there should be a "b" in the file mode, but that's unrelated to having multiple FILE*
+09:57 < jtanx> yep then I think that's due to you running it on a multicore processor
+09:57 < jtanx> maybe
+09:57 < sam_moore> I think the OS locks around file access are based on PID
+09:58 < jtanx> on linux all file locks are advisory only
+09:58 < sam_moore> Can
+09:58 < sam_moore> Anyway, it's clearly safest to just have one FILE*
+09:58 < jtanx> yeah
+09:58 < jtanx> I'm changing it now
+10:09 < jtanx> oh ffs that's the second time it's crashed simply because I didn't do make clean before make
+10:16 < sam_moore> haha
+10:17 < sam_moore> Modify the make file to always run make clean?
+10:28 < jtanx> haha that's one option
+10:28 < jtanx> it's because make doesn't consider changes to the header files
+10:28 < jtanx> there's a way to make it detect those, but *effort*
+10:40 < sam_moore> Regarding Actuators, I think if we design for a thread to control (one or more) actuators that will make it easy to add more advanced control later
+10:42 < sam_moore> eg: The Actuator_Handler just sets some kind of formula or says "Every time sensor 0 is stable for Xs (within a range) change actuator by Y" and the thread watches the sensor data and changes the actuator when necessary
+10:43 < sam_moore> You could also add feedback that way too
+10:44 < sam_moore> "Set this actuator so that sensor 0 has value X"
+10:46 < jtanx> hmm
+10:47 < jtanx> about the sensor stuff
+10:47 < jtanx> there were a few things I noticed
+10:48 < jtanx> the time wrapping
+10:48 < jtanx> could still give a negative start_time or end_time
+10:48 < jtanx> if the user specified a very negative start_time/end_time
+10:49 < jtanx> (eg specifies a negative time greater than the total running time)
+10:49 < sam_moore> Oh yeah
+10:49 < jtanx> so in data_printbytimes
+10:49 < jtanx> I just clamped it to zero
+10:49 < jtanx> instead of asserting
+10:49 < sam_moore> Fair enough
+10:49 < jtanx> strangely this code seems to work pretty good:
+10:49 < jtanx> void Data_PrintByTimes(DataFile * df, double start_time, double end_time, DataFormat format)
+10:49 < jtanx> {
+10:49 < jtanx> assert(df != NULL);
+10:49 < jtanx> //Clamp boundaries
+10:49 < jtanx> if (start_time < 0)
+10:49 < jtanx> start_time = 0;
+10:50 < jtanx> if (end_time < 0)
+10:50 < jtanx> end_time = 0;
+10:50 < jtanx> int start_index = 0, end_index = 0;
+10:50 < jtanx> if (start_time < end_time)
+10:50 < jtanx> {
+10:50 < jtanx> start_index = Data_FindByTime(df, start_time, NULL);
+10:50 < jtanx> end_index = Data_FindByTime(df, end_time, NULL);
+10:50 < jtanx> }
+10:50 < jtanx> Data_PrintByIndexes(df, start_index, end_index, format);
+10:50 < jtanx> }
+10:50 < jtanx> oh but that was the other ting
+10:50 < jtanx> it works better if you have exclusive end index/time
+10:50 < sam_moore> Ok
+10:50 < sam_moore> I have to go now
+10:50 < jtanx> ok
+10:50 < sam_moore> See you this afternoon if you're still there
+10:50 < jtanx> yep see you then
+10:51 < jtanx> probably
+13:44 < callum> should i just add the sensor threshold values to the sensor struct?
+13:47 < jtanx> probably not
+13:47 < jtanx> I think you have to create a new struct
+13:48 < jtanx> are you at uni?
+13:49 < callum> yea. why is a new struct necessary?
+13:49 < jtanx> come to g19
+13:49 < jtanx> the bbb is setup too
+16:17 < sam_moore> Are you guys still in G19?
+16:17 < sam_moore> On my way now
+18:42 < jtanx> lol you need to use a transistor to turn on an led from the gpio pins
+19:56 < jtanx> ok I think I'll just escape all input parameters from within the request loop
+19:56 < jtanx> then at least you're guaranteed that user input won't stuff up anything
+19:56 < jtanx> (as in making it generate invalid JSON)
+20:53 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 23.0.1/20130814063812]"]
+--- Day changed Sat Sep 14 2013
+11:32 < sam_moore> blergh, too many different versions of OpenCV
+11:32 < sam_moore> Also too many different data types
+11:33 < sam_moore> "Getting a digital image into numbers can be very handy for your image processing. In this post i will brief just that."
+11:34 < sam_moore> Thank god, finally a tutorial that makes sense
+11:34 -!- Irssi: #mctxuwa_softdev: Total of 2 nicks [0 ops, 0 halfops, 0 voices, 2 normal]
+11:34 < sam_moore> ... no one else is in the channel
+11:34 < sam_moore> :S
+12:36 < jtanx> on start/stop/pausing an experiment
+12:36 < jtanx> currently everything is relative to g_options.start_time
+12:42 < sam_moore> Yeah
+12:43 < sam_moore> Perhaps make an ExperimentTime function and that leaves us free to change what the time stamps are relative to?
+12:43 < sam_moore> Looking into this interferometer stuff...
+12:43 < jtanx> ok, maybe as part of the Control_ module then
+12:43 < sam_moore> Interesting problem, I think I've worked out an algorithm that isn't too terrible
+12:44 < jtanx> yeah?
+12:44 < jtanx> how does it work
+12:45 < sam_moore> If you have a sinusoidal wave, find the x intercepts, and that lets you calculate the phase and frequency
+12:45 < sam_moore> That's the simple version
+12:46 < jtanx> haha alright
+12:46 < sam_moore> I have 3 pages of notes that I can show Adrian/Adam if I can't get it working
+12:46 < jtanx> wow
+12:46 < jtanx> nice work
+12:47 < sam_moore> Fortran would be good for the array operations
+12:47 < sam_moore> If fortran were able to do anything *other* than array operations it would be a nice language...
+12:47 < jtanx> have you ever used Fortan before?
+12:48 < sam_moore> Yes
+12:48 < sam_moore> I'm going to keep it in C though
+12:48 < sam_moore> It took long enough to get OpenCV working
+12:48 < sam_moore> I'm sure there's some kind of convoluted Fortran library
+12:49 < sam_moore> But a seperate fortran program won't talk to our server
+12:49 < jtanx> can you compile the code as a library with C interop
+12:49 < sam_moore> Erm..
+12:49 < sam_moore> Probably more effort than just doing it in C
+12:49 < jtanx> probably
+12:51 < jtanx> I think a separate mutex is needed around the *_Stop() functions
+12:51 < jtanx> because if a client sends in a request
+12:51 < sam_moore> Oh, good point
+12:52 < jtanx> although that's only going to be a problem if our own software calls the stop function because of the sanity checks
+12:52 < sam_moore> If there are 2 requests?
+12:52 < sam_moore> No wait, the FCGI stuff is serial
+12:52 < jtanx> the request loop is single threaded
+12:52 < sam_moore> It has to finish processing one request to respond to the next
+12:52 < sam_moore> That's fine
+12:56 < sam_moore> Put a mutex in the Sensor struct, use the same one that's already in the Actuator struct, put critical sections around setting the "s->record_data" or "a->activated" variables (or whatever you choose to use as a "is thing running" flag)
+12:56 < sam_moore> And check the flag hasn't already been set before doing anything
+12:57 < sam_moore> Similar to how the Thread_Running() and Thread_QuitProgram() functions used to work
+12:57 < sam_moore> .... Before I deleted them
+12:57 < sam_moore> Any thread could call Thread_QuitProgram, and repeated calls would have no effect
+13:01 < jtanx> ok I'll try that
+13:51 < jtanx> actually, what about this:
+13:51 < jtanx> in control.c, it maintains its own mutex
+13:51 < jtanx> so
+13:52 < jtanx> and it's the only place where Sensor_Start/Stop Actuator_Start/Stop is called
+13:53 < jtanx> You then have Control_Lock/Unlock that's placed around the responses in the handler functions
+13:53 < jtanx> those functions internally lock/unlock the mutex held in control.c
+13:53 < jtanx> ah stuff it
+13:53 < jtanx> I'll commit it to my git repo
+13:54 < jtanx> I still don't know how the pause functionality should be done though
+13:54 < jtanx> especially with the time discontinuities
+14:01 < jtanx> oh yeah, just a reminder that you have to be careful using FCGI_LONG_T vs FCGI_INT_T
+14:02 < jtanx> if your variable is of type int but you give FCGI_LONG_T, and where sizeof(int) != sizeof(long) then there could be a buffer overflow
+14:13 < sam_moore> Right
+14:14 < sam_moore> If you want to commit to the main repo you can use a new branch
+14:14 < sam_moore> Having a single mutex in control.c instead of one per sensor/actuator is probably better
+14:15 < sam_moore> With the pause functionality... I think if you just don't call Data_Close() (and don't call Data_Open on resuming) but otherwise stop the threads running Sensor_Loop in the same way
+14:15 < sam_moore> That should do most of it
+14:16 < sam_moore> Don't reset the experiment start time
+14:17 < sam_moore> Perhaps the experiment start time needs to be stored somewhere though
+14:18 < sam_moore> Anyway, I'll let you solve the problem :P
+14:48 < jtanx> ._.
+15:45 < jtanx> well I think I got something semi-working
+15:45 < jtanx> it involved adding on Sensor_PauseAll/ResumeAll + Acuator_PauseAll/ResmueAll (plus the individual Pause/Resume) functions
+15:46 < sam_moore> Cool
+15:46 < sam_moore> I think I'm starting to get my head around OpenCV
+15:46 < sam_moore> The documentation for it is a bit crap
+15:46 < jtanx> oh don't get me started on opencv documentation
+15:47 < jtanx> yesterday it took me and callum around 30 mins to figure out that CvMat structure
+15:47 < sam_moore> Haha
+15:47 < sam_moore> It probably took me a bit longer :S
+15:47 < jtanx> the problem is that most of it's geared towards c++ usage
+15:49 < jtanx> back on that control stuff, basically those Sensor/Actuator_[Start|Stop|Pause|Resume] functions can only be called via the Control_[Start|Stop|Pause|Resume] functions
+15:50 < jtanx> partly because of threading issues but also because it's the only place where the current state is currently kept track of
+15:51 < jtanx> (e.g if you call Sensor_Start twice then...
+15:53 < sam_moore> Seems sensible
+15:53 < jtanx> anyway, I'm taking a break from this stuff for a while
+15:53 < sam_moore> Yeah, fair enough
+15:54 < jtanx> If you want to see what I've done so far, it's on my git repo (haven't added comments yet)
+15:54 < sam_moore> Cool
+16:50 < sam_moore> The IplImage vs CvMat conversion is dumb
+16:50 < sam_moore> In fact I don't think you even need to do it?
+16:51 < sam_moore> Well at least to display an image you can just pass a CvMat
+16:51 < sam_moore> Maybe you still need the IplImage to capture from a camera
+16:51 < sam_moore> I worked out how to convert from IplImage to CvMat anyway
+16:53 < sam_moore> Other than that, OpenCV doesn't actually seem horrible to use
+16:53 < sam_moore> Just... contradictory documentation
+16:55 < sam_moore> Anyway... I've created a moving sinusoidal image!
+16:56 < sam_moore> Now to work out why the algorithm keeps returning -nan :S
+16:56 < sam_moore> Also for some reason the image is blue instead of red
+16:56 < sam_moore> But whatever
+16:57 < jtanx> :S
+16:57 < jtanx> BGR vs RGB?
+17:02 < sam_moore> Looks like it
+17:03 < sam_moore> Doing this with an actual camera is going to suck
+17:04 < sam_moore> See, I've worked out an algorithm to cope with changing background light conditions
+17:04 < sam_moore> Because differentiating a sinusoid gives you another sinusoid with the same phase (offset by pi/2)
+17:05 < sam_moore> Buut... differentiating with finite differences adds more noise
+17:05 < sam_moore> Or rather it makes the noise more pronounced
+17:07 < sam_moore> Hopefully sensors is considering this
+17:07 < sam_moore> If they want to shoot a laser into a camera, they should really do it in a way that keeps out background light
+17:07 < sam_moore> Buuut
+17:07 < sam_moore> I think they mentioned using one camera to do both the interferometer and look at the can
+17:07 < sam_moore> :S
+17:08 < sam_moore> Oh well, the optics is their problem
+17:09 < sam_moore> I suppose I will prepare a report or something about the algorithm and what conditions the image needs to satisfy
+17:12 < jtanx> um
+17:12 < jtanx> there's going to be two cameras
+17:12 < jtanx> because it was too much hassle moving it
+17:12 < sam_moore> Yes... but one was meant to be looking at each can?
+17:13 < jtanx> oh
+17:13 < jtanx> hmm
+17:13 < sam_moore> We should ask them
+17:13 < jtanx> I just thought that one would be for the interferometer (no need to look at can) and the other would be for the can to be blown up
+17:13 < sam_moore> Probably
+17:15 < jtanx> but I was thinking for the one to be blown up
+17:16 < jtanx> you could possibly just stream it using something like ffserver
+17:16 < jtanx> instead of passing it through opencv
+17:16 < sam_moore> Yeah, that seems like it's probably easier
+17:23 -!- jtanx [
[email protected]] has quit ["my antivirus is pestering me to restart"]
+18:19 < sam_moore> So... when you take an image with a camera the pixels are stored as rgb with 0->255 values
+18:20 < sam_moore> You can convert the IplImage to a CvMat and it keeps the values ordered as rgb with 0->255 values
+18:20 < sam_moore> And then...
+18:20 < sam_moore> If you try and display it...
+18:20 < sam_moore> The display function treats it as having values in bgr order with 0->1 values
+18:20 < sam_moore> -_-
+18:21 < sam_moore> (So unless you manually adjust all the values you'll just get a white picture in the display)
+18:54 < jtanx> hahaha
+18:54 < jtanx> what
+18:56 < jtanx> for cvShowImage:
+18:56 < jtanx> If the image is 8-bit unsigned, it is displayed as is.
+18:56 < jtanx> If the image is 16-bit unsigned or 32-bit integer, the pixels are divided by 256. That is, the value range [0,255*256] is mapped to [0,255].
+18:56 < jtanx> If the image is 32-bit floating-point, the pixel values are multiplied by 255. That is, the value range [0,1] is mapped to [0,255].
+18:57 < jtanx> so it's treating it as 32-bit floating point?
+20:41 < sam_moore> I think it's because of how I converted the IplImage to a CvMat
+20:42 < sam_moore> My algorithm is almost working
+20:43 < sam_moore> The fact that angles wrap around every 2*pi is giving me a headache though
+20:43 < sam_moore> This is pretty fun
+20:45 < sam_moore> Hopefully I can do Computer Vision as an optional, I think its in the list
+20:45 < sam_moore> Maybe I shouldn't make all my optionals CS units, but meh
+21:20 < jtanx> :P
+21:20 < jtanx> the computer vision unit sounds interesting, but a lot of maths/theory involved
+22:02 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 23.0.1/20130814063812]"]
+--- Day changed Sun Sep 15 2013
+11:02 < sam_moore> I'm really glad I put the IRC logs under git
+11:02 < sam_moore> It's very helpful for keeping my diary up to date
+11:40 < jtanx> hehe
+11:40 < jtanx> I'm gladful for git
+11:48 < jtanx> about the bbb
+11:48 < jtanx> it's probably best to just install debian to the internal memory
+11:54 < jtanx> on the controls
+11:54 < jtanx> maybe we can have the following states: start/pause/resume/stop/close
+11:56 < jtanx> actually, maybe just start/pause/resume/close
+11:56 < jtanx> when an 'emergency stop' due to the sanity checks occurs
+11:57 < jtanx> actually, start/pause/resume/close/emergency
+11:57 < jtanx> start/pause/resume/close can only be initiated by the client
+11:58 < jtanx> emergency can be initiated by client/server
+11:58 < jtanx> in emergency, actuators are set as necessary then deactivated
+11:58 < jtanx> but importantly, the data files remain open, which hopefully removes the need for mutexes
+12:26 < sam_moore> That seems sensible
+13:20 < sam_moore> Hey
+13:20 < sam_moore> It's actually quite simple to stream an image through FastCGI
+13:21 < jtanx> yeah
+13:21 < jtanx> you just fwrite the image buffer
+13:21 < sam_moore> Yep
+13:21 < jtanx> I had some test code for that but could never test it
+13:22 < jtanx> no webcam
+13:22 < sam_moore> Ah, damn
+13:22 < sam_moore> I've written some code, it works pretty brilliantly
+13:22 < jtanx> nice
+13:22 < jtanx> this control code is really annoying me
+13:23 < jtanx> just all the considerations of if you need a mutex here or not and 'what happens if...'
+13:23 < sam_moore> Ah, sorry to hear that
+13:23 < jtanx> yeah nah its ok
+13:24 < jtanx> but that interferometer stuff you did looks really cool
+13:26 < sam_moore> Yeah, it worked surprisingly well
+13:26 < sam_moore> It can cope with a fair bit of background light, even without all the improvements I was thinking about
+13:27 < sam_moore> If I add 2 or 3 more columns to scan, and use the fact that we actually know the spatial frequency to eliminate nodes that are too close together
+13:27 < sam_moore> It should be pretty good
+13:27 < sam_moore> I'm not sure what the laser intensity will be, but the generated image looks similar to ones you can find on the internet
+13:28 < sam_moore> One possible problem is if the mirrors aren't flat enough
+13:28 < sam_moore> But hopefully it won't be a big issue (since they've pretty much just specified an expensive pre-built kit which presumably has flat mirrors)
+13:29 < jtanx> yeah
+13:30 < jtanx> here's to hoping 'it just works'
+13:30 < sam_moore> Yep :P
+13:31 < sam_moore> Got to go, bye
+13:32 < jtanx> ok cya
+15:56 -!- jtanx_ is now known as jtanx
+21:48 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 23.0.1/20130814063812]"]
+--- Day changed Mon Sep 16 2013
+16:05 -!- jtanx_ [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 23.0.1/20130814063812]"]
+19:44 < jtanx> no mctxbot while I work on my f# project
+22:58 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 23.0.1/20130814063812]"]
+--- Day changed Tue Sep 17 2013
+09:50 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 23.0.1/20130814063812]"]
+15:34 < jtanx> to run the software not as root, we need to at least add it to the 'video' group so it can access the webcam
+15:34 < jtanx> not sure about the sensors
+15:34 < jtanx> eg sudo usermod -a -G video debian
+15:38 < sam_moore> Ah, cool
+15:39 < sam_moore> I think I worked out how to get around the camera crashing without killing the rest of the program, won't implement it for a while though
+15:57 -!- jtanx_ is now known as jtanx
+16:46 -!- jtanx_ is now known as jtanx
+18:02 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 23.0.1/20130814063812]"]
+23:18 < Callum> hey
+23:23 < sam_moore> Hi
+23:24 < Callum> do you know how to do the second question of that assignment?
+23:24 < Callum> https://docs.google.com/file/d/0B241H5liqVlnbHZETXRZX1Mwd1k/edit?usp=sharing
+23:25 < Callum> rather remember
+23:35 < sam_moore> Well, you can eliminate any epsilon^2 when you substitute those formulae for v into the formula for gamma
+23:35 < sam_moore> Then it has a really simple taylor expansion
+23:35 < Callum> i might have to hunt you down tomorrow to show me. or maybe il be able to think straight tomorrow.
+23:38 < sam_moore> You can express one of the epsilons in terms of the other one after that
+23:40 < sam_moore> I'm pretty busy tomorrow
+23:42 < Callum> mhmm. so no free time at all? coz im fairly free
+23:49 < sam_moore> Not from 8:00am until 5:00pm
+23:49 < Callum> ohwow. you are busy
+23:49 < Callum> im still unsure what im meant to be applyign the taylor expansion to
+23:53 < sam_moore> Substitute in the suggested formulae to gamma, simplify it, then apply a taylor expansion
+23:53 < sam_moore> Anyway, goodnight, good luck
+23:53 < Callum> night
+--- Day changed Wed Sep 18 2013
+09:11 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 23.0.1/20130814063812]"]
+21:03 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 24.0/20130910160258]"]
+--- Day changed Thu Sep 19 2013
+16:08 < jtanx> one thing I forgot to mention - the latest version on git uses syslog
+16:08 < jtanx> if you copy the config file (server-configs/rsyslog.d/* ??) to /etc/
+16:09 < jtanx> it will log to /var/log/mctx[something I can't remember].log
+16:10 < jtanx> i'm pretty sure you can also create a log file specifically for warning level and above (so it logs in three places, stderr, the normal log, and a special 'error' log), but I haven't set this up yet
+18:17 < jtanx_> :0
+18:22 < sam_moore> ?
+18:50 -!- jtanx_ is now known as jtanx
+19:52 -!- MctxBot_ is now known as MctxBot
+20:24 < jtanx> I got the camera image to be updated in javascript instead: http://mctx.us.to:8080/test/
+20:25 < jtanx> right now it's updated around every ~400ms because my webcam is running through a usb1.1 link which seriously limits how fast it can update
+20:36 < jtanx> (its running on a pentium 3 computer)
+21:35 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 24.0/20130910160258]"]
+--- Day changed Fri Sep 20 2013
+18:02 < jtanx> this seems like an interesting option, at least for the cam that just shows the can explode:http://sourceforge.net/apps/mediawiki/mjpg-streamer/index.php?title=Main_Page
+18:51 < sam_moore> Cool
+18:51 < sam_moore> It, ah
+18:52 < sam_moore> Looks like we might have to recompile the kernel to get PWM working
+18:52 < sam_moore> Hooray
+18:52 < sam_moore> Kernel compiling
+18:52 < sam_moore> What could possibly go wrong?
+19:02 < jtanx> really??
+19:02 < jtanx> why....
+19:03 < jtanx> no am33xx_pwm module?
+19:04 < sam_moore> Not that I can tell
+19:05 < jtanx> well crap
+19:06 < sam_moore> Well... surely I can find a precompiled kernel somewhere
+19:06 < jtanx> it seems that stuff for the BBB is at a really premature stage
+19:07 < sam_moore> Yeah, RPi is much nicer
+19:07 < jtanx> the BBB is just too new..
+19:07 < sam_moore> What's the point of having fancy things like ADC and PWM...
+19:07 < sam_moore> If you can't actually use them without jumping through hoops
+19:07 < jtanx> hahaha
+19:07 < jtanx> were you referencing this, when you said we might have to recompile the kernel? https://groups.google.com/d/msg/beagleboard/wjbOVE6ItNg/AGYVRhYbTX8J
+19:08 < sam_moore> Yeah
+19:08 < sam_moore> ...
+19:08 < sam_moore> I wonder if I can take the kernel used by Angstrom and stick it in /boot
+19:08 < sam_moore> on the SD card
+19:09 < jtanx> ._.
+19:10 < jtanx> what about kernels from here
+19:10 < jtanx> http://rcn-ee.net/deb/precise-armhf/
+19:10 < jtanx> sorry
+19:10 < jtanx> http://rcn-ee.net/deb/wheezy-armhf/
+19:11 < jtanx> what's it currently running
+19:12 < sam_moore> 3.8.13-bone20
+19:13 < sam_moore> Try 3.8.13-bone28 ?
+19:15 < jtanx> what about v3.8.9-bone15
+19:15 < jtanx> oh wait
+19:15 < jtanx> ~.~
+19:15 < sam_moore> I wonder why the BBB developers didn't go with debian...
+19:15 < jtanx> v3.11.0-bone5
+19:15 < jtanx> yeah
+19:15 < jtanx> why angstrom, of all things
+19:15 < sam_moore> Is angstrom used anywhere else?
+19:16 < jtanx> dunno
+19:16 < sam_moore> Other embedded devices, heh
+19:16 < sam_moore> Everyone just has to use their own distro :P
+19:17 < sam_moore> I didn't see the .11 kernels, I'll try the latest one
+19:17 < jtanx> want to try the rc version? lol
+19:18 < jtanx> oh the rc versions are older than .11
+19:19 < jtanx> how does it work?
+19:19 < jtanx> do you just download from http://rcn-ee.net/deb/wheezy-armhf/v3.11.0-bone5/
+19:19 < jtanx> the .sh script and run it from the BBB?
+19:24 < sam_moore> I think so
+19:25 < sam_moore> Well... here goes nothing
+19:26 < sam_moore> Hopefully between 3.8 and 3.11 they actually enabled PWM by default...
+19:27 < sam_moore> It looks like install-me.sh downloads all the .deb files appropriately
+19:29 < sam_moore> It's creating files like: /lib/firmware/bone_pwm_P8_36-00
+19:29 < sam_moore> Which looks promising
+19:29 < sam_moore> Snoopy is not yet on fire
+19:29 < sam_moore> Which is promising
+19:29 < sam_moore> Rebooting and praying
+19:31 < sam_moore> And... it booted into 3.8.13-bone20 -_-
+19:31 < sam_moore> Tempted to just symlink that filename to the new kernel
+19:34 < sam_moore> The new kernel exists in /boot, but obviously there's some boot config that needs to get set
+19:39 < sam_moore> Ok, screw this, I'm making a symlink just to see if it works
+19:40 < jtanx> lol
+19:41 < jtanx> no grub
+19:43 < jtanx> did update-initramfs run?
+19:46 < jtanx> or maybe we need mkimage
+19:47 < jtanx> yeah probably need mkimage
+19:50 < sam_moore> Um, zImage is a symlink to the kernel
+19:50 < sam_moore> But regardless, the new one won't boot
+19:50 < sam_moore> Such a pain
+19:50 < sam_moore> I wonder if we can just toggle the GPIO pin fast enough to implement PWM
+19:56 < sam_moore> Using a decent oscilloscope, I read a 6us period when switching GPIO1_28 on/off as fast as possible
+19:59 < sam_moore> That
+19:59 < sam_moore> s like 166KHz
+19:59 < sam_moore> Heh
+19:59 < sam_moore> Maybe the sources you found were closing the file and reopening it?
+20:01 < sam_moore> Yeah, wow
+20:02 < sam_moore> Using fopen, fprintf and fclose each time takes the period to 224us
+20:02 < sam_moore> Or 4KHz
+20:03 < sam_moore> Also for future reference, that CRO in G19 is definitely broken; it's always a lovely square wave on the modern oscilloscope I'm testing with now
+20:12 < jtanx> haha ok
+20:13 < jtanx> but without kernel support your pwm signal won't be very accurate
+20:13 < sam_moore> Yeah, I suppose
+20:13 < jtanx> how come the new kernel won't boot?
+20:14 < sam_moore> No idea
+20:14 < jtanx> Hmm
+20:14 < jtanx> this lack of documentation on how you do such procedures is pretty crap
+20:15 < sam_moore> Yeah, kernel developers aren't great on documentation
+20:15 < jtanx> is the mkimage package present?
+20:15 < jtanx> if not, install it and try running the install-me script again
+20:16 < jtanx> from this: http://www.jeremycole.com/blog/2011/03/09/beagleboard-upgrading-from-debian-5-to-debian-6/
+20:16 < jtanx> it should be just running that install script and rebooting
+20:16 < sam_moore> Alright, uboot-mkimage I presume?
+20:16 < jtanx> i guess
+20:17 < jtanx> and update-initramfs?
+20:18 < jtanx> (just looking at what the install-me.sh script uses)
+20:18 < sam_moore> Oh, does the script not fail if packages it needs don't exist...
+20:18 < sam_moore> -_-
+20:19 < jtanx> a quick scan says nup
+20:24 < sam_moore> Already had initramfs-tools but not mkimage, so I'll try again
+20:27 < sam_moore> I should probably have focused on the ADC reading before PWM
+20:27 < sam_moore> We're definitely going to get asked about aliasing again
+20:28 < jtanx> urgh yeah
+20:28 < sam_moore> I don't have a signal generator here though
+20:28 < jtanx> this BBB has definitely not been designed with 'plug n play' in mind
+20:28 < sam_moore> Mmm
+20:29 < sam_moore> The Angstrom image would work for this stuff, but obviously has limitations in terms of the webserver stuff
+20:29 < sam_moore> But even with Angstrom you still have to go through a similar convoluted process to control pins
+20:30 < sam_moore> From what I can tell there are two ways to do it
+20:30 < sam_moore> SYSFS, which I can't find much documentation on
+20:30 < sam_moore> Which is the /sys/class/gpio/ stuff
+20:30 < sam_moore> Which actually seems more intuitive than the other way
+20:31 < sam_moore> That involves echoing a bunch of cruft to /sys/devices/cape-manager/slots/ or something similar
+20:31 < jtanx> hmm
+20:32 < sam_moore> There is a /sys/class/pwm
+20:32 < sam_moore> But unfortunately whatever you export it complains about
+20:32 < sam_moore> Probably because the kernel wasn't compiled with it enabled
+20:32 < jtanx> is this with the old kernel?
+20:32 < sam_moore> Yeah
+20:33 < sam_moore> Hangon, the new one's probably finished building by now
+20:34 < jtanx> we don't know if the new one has been compiled with pwm support though
+20:34 < sam_moore> Yeah
+20:36 < sam_moore> The diff between the config options for the kernels shows that the old one has a comment "CONFIG_EHRPWM_TEST is not set" and the newer one has no reference to it
+20:37 < sam_moore> ...
+20:37 < sam_moore> Someone at some point
+20:37 < sam_moore> Has realised that PWM was not enabled
+20:37 < sam_moore> And commented that it isn't enabled
+20:37 < sam_moore> WHY THE HELL DIDN'T THEY JUST ENABLE IT
+20:37 < jtanx> :P
+20:37 < sam_moore> Anyway it still booted off the old kernel
+20:37 < sam_moore> Dinner time
+20:38 < jtanx> ok
+21:15 < sam_moore> http://www.eewiki.net/display/linuxonarm/BeagleBone+Black#BeagleBoneBlack-LinuxKernel
+21:15 < sam_moore> Looks promising
+21:16 < sam_moore> Hmm...
+21:17 < sam_moore> I'll try using 3.8 but building with the PWM
+21:18 < jtanx> building from source - fun fun :P
+21:19 < sam_moore> Well in theory I just change a config option
+21:19 < jtanx> yeah
+21:19 < sam_moore> When that doesn't work and I have to modify the source, that's when the fun begins
+21:19 < jtanx> let's just hope it works
+21:19 < sam_moore> Yeah, if it doesn't we're (beagle) boned
+21:19 < jtanx> oh while you're at it, figure out how to enable the ethernet-over-usb feature
+21:19 < sam_moore> Haha
+21:20 < jtanx> loljk
+21:25 < sam_moore> Hmmm... compiling a kernel is going to take a while
+21:26 < sam_moore> Ergh, why am I running it on this Pentium D
+21:30 < sam_moore> Hmmm, more troubling... why does a debian wheezy server have OpenSUSE sources in sources.list
+21:30 < sam_moore> Oh well, not my problem
+21:33 < jtanx> heh
+21:33 < sam_moore> How the hell are we going to explain this in the progress report...
+21:34 < sam_moore> Also, I didn't fully understand, why can't you use the same image for multiple BBB?
+21:34 < sam_moore> Are we going to have to do this all again for the other BBB?
+21:34 < sam_moore> Spike I mean
+21:37 < jtanx> no idea
+21:38 < jtanx> without some sort of debugging cable to see what happens when it boots, who knows
+21:38 < sam_moore> :S
+21:39 < sam_moore> I love how git gets to the head of the branch
+21:39 < sam_moore> By starting at the initial commit
+21:39 < sam_moore> And going through every commit and changing the file
+21:42 < sam_moore> It hasn't started building yet
+21:42 < sam_moore> And the way you customise the build...
+21:42 < sam_moore> Is to build it with the defaults, so that the options file exists, then change the options, then build it again -_-
+21:43 < jtanx> ಠ ಠ
+21:43 < sam_moore> Oh well, I have to go home, I'll try finish this tomorrow
+21:43 < sam_moore> Bye
+21:43 < jtanx> ok
+21:43 < jtanx> bye
+23:20 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 24.0/20130910160258]"]
+--- Day changed Sat Sep 21 2013
+11:23 < sam_moore> http://hipstercircuits.com/enable-pwm-on-beaglebone-with-device-tree-overlays/
+11:23 < sam_moore> So... according to this I should add pwm_test as a kernel module
+11:24 < sam_moore> "This is it. It is now three o’clock in the morning and I have not slept or eaten in two days. My neck is sore, my ass has fallen asleep and my organs are screaming “slow down, man”. I no longer see [CC]s, [LD]s and [AS]s, I only see blondes, brunettes and redheads flashing before my eyes. With my last ounce of energy I barely manage to type in “reboot” as my face hits the keyboard. And that is when it happens.."
+11:25 < sam_moore> Ummm
+11:25 < sam_moore> It's awesome that this guy has solved the problem (I think)
+11:26 < sam_moore> But a bit depressing that it still isn't in the official kernel
+11:29 < sam_moore> I think most people just give up and use Angstrom, which is tempting
+11:30 < sam_moore> I still have that HTTP server code... :P
+11:45 < sam_moore> Looks like Robert C Nelson's 3.8 kernel does have a pwm_test module
+11:45 < sam_moore> Maybe the image you used just had an out of date kernel?
+11:45 -!- Irssi: #mctxuwa_softdev: Total of 3 nicks [0 ops, 0 halfops, 0 voices, 3 normal]
+11:51 < jtanx> hmm
+11:51 < jtanx> no idea
+11:51 < jtanx> it was made in july I think and it uses those rcn kernels
+11:51 < jtanx> We could always use lighttp
+11:51 < jtanx> lighttpd on angstrom if necessary
+11:52 < jtanx> lighttpd and install mod_fastcgi
+11:55 < jtanx> ok so the image uses 3.8.13-bone20, so maybe it wasn't enabled in that version, but it now is in 3.8.13-bone28?
+12:02 < sam_moore> I've just built 3.8.13-bone28 and the module pwm_test exists
+12:03 < sam_moore> ... I also copied the code from that guy's blog as pwm_test2 just in case :P
+12:03 < sam_moore> I'll have to test it later, but at least we have the kernel module
+12:08 < jtanx> nice
+12:39 < jtanx> ohhhhh I know why it didn't work from one bbb to the next, using the same image
+12:39 < jtanx> When you boot for the first time, it assigns the ethernet port eth0
+12:39 < jtanx> when you then take it out and boot it on a different BBB
+12:40 < jtanx> because the ethernet device has a different id, it gets assigned to say eth1
+12:40 < jtanx> and because you don't have any config for eth1 in your network config, there's no internet access
+12:40 < jtanx> http://www.eewiki.net/display/linuxonarm/BeagleBone#BeagleBone-Networking:UsingasharedSDcardwithMultipleBeagleBone
+12:41 < jtanx> should fix that
+15:10 < jtanx> Eh, you know what I'll just stop the threads when you want to pause it
+15:10 < jtanx> instead of conditionals
+15:11 < jtanx> it's not like you're pausing the experiment that often
+15:18 < sam_moore> That's fine
+15:19 < sam_moore> The conditional approach is only really necessary if you're constantly pausing the threads
+15:19 < sam_moore> I used it for an nbody simulator, so the computation of force and position was split up between threads
+15:19 < sam_moore> It's really slow if you have to stop and then recreate the threads on every step
+15:22 < sam_moore> Although still actually faster than the single threaded program
+15:22 < sam_moore> Well, depending on how many objects you simulated
+15:23 < sam_moore> Anyway, just stop the threads, it's simpler to code and the performance effect in our case is probably negligable
+15:30 < jtanx> yeah
+15:30 < jtanx> say you had an actuator that was being controlled at that instant when an 'emergency stop' was issued
+15:31 < jtanx> since an 'emergency stop' is the same as just pausing (eg stopping the thread but keep DataFile open)
+15:31 < jtanx> you'd have to wait for that action to be completed before the 'emergency stop' would be propagated
+15:34 < jtanx> welp I guess that's why there's hardware safety interlocks
+15:38 < jtanx> Also, technically wouldn't it be possible to try to set the actuator value before the current value has been set
+15:38 < jtanx> Since there's no queue
+15:39 < jtanx> a->control could be overwritten by a new request as Actuator_SetValue operates
+16:12 < sam_moore> We want that right?
+16:13 < sam_moore> I'll look at it later if I get time
+16:14 < jtanx> I don't know if we want that or not
+16:15 < jtanx> wait want as in the current behaviour or the behaviour with a queue?
+16:16 < sam_moore> The current behaviour
+16:16 < sam_moore> I don't think you need a queue
+16:16 < sam_moore> You can extend the critical section in Actuator_Loop to stop the current control getting overwritten
+16:17 < sam_moore> Move the pthread_mutex_unlock on line 121 to below line 127 (Actuator_SetValue)
+16:17 < sam_moore> That way if Actuator_SetControl is called before the value has been successfully set, it will just wait
+16:17 < sam_moore> Mutexes actually implement a queue
+16:18 < sam_moore> If one thread has a lock on the mutex, subsequent threads that try to access the mutex will queue up; whenever the mutex is unlocked the next thread (if any) which was waiting will get it
+16:18 < jtanx> ok
+16:23 < jtanx> I'll leave it as is for now
+16:23 < sam_moore> Sure
+16:49 < sam_moore> PWM working
+16:49 < jtanx> nice
+16:50 < jtanx> I still don't really understand - did you compile the kernel from scratch
+16:50 < jtanx> or did you figure out how to use the install-me.sh script
+16:50 < sam_moore> I did, but I didn't need to modify it
+16:50 < jtanx> huh
+16:50 < jtanx> ok
+16:51 < sam_moore> http://www.eewiki.net/display/linuxonarm/BeagleBone+Black#BeagleBoneBlack-LinuxKernel
+16:52 < jtanx> so if we do this: http://www.eewiki.net/display/linuxonarm/BeagleBone+Black#BeagleBoneBlack-Networking:UsingasharedSDcardwithMultipleBeagleBone
+16:52 < jtanx> We should be able to just copy our image
+16:52 < jtanx> and stick it on the electronics' BBB
+16:53 < sam_moore> Sounds good
+16:53 < sam_moore> I'm glad that worked
+16:53 < jtanx> yeah
+16:54 < jtanx> I wonder if it also enabled the usb0 stuff (ethernet over usb)
+16:58 < sam_moore> PWM appears to have picosecond resolution? Or at least the arguments are in ps
+17:02 < jtanx> oO
+17:11 < sam_moore> ADC can sample at ~4KHz
+17:11 < sam_moore> But... that's using bash
+17:11 < sam_moore> It will probably be massively fast when done in C
+17:11 < jtanx> um
+17:11 < jtanx> is there any need to have it sample at a consistent rate
+17:11 < jtanx> as in, with threads there's no guarantee
+17:12 < jtanx> of a consistent sampling rate
+17:12 < sam_moore> Yes, you're right
+17:13 < sam_moore> I don't think we can achieve a consistent sampling rate, but I don't think it's critical that we do
+17:14 < sam_moore> As soon as we make our software run in an operating system with a kernel and other processes that can run as well, it gets pretty unfeasable to have a constant sample rate
+17:14 < sam_moore> We can have it constant to within an uncertainty I guess
+17:15 < jtanx> yeah, true
+17:18 < sam_moore> If you wanted a really high constant sample rate (say much faster than 1us which is probably the best case we could get) you'd have to use a more low level embedded device
+17:18 < sam_moore> Well I guess you could compile your own kernel for the BBB
+17:19 < sam_moore> But either way you'd have to physically run the webserver/GUI interface stuff on a seperate device
+17:19 < sam_moore> At this stage my feeling is what we have is good enough given the complexity of all the requirements we were given
+17:23 < jtanx> yeah
+17:25 < sam_moore> Hmm, I can set some GPIO pins to toggle whenever Sensor_Read is called and get an idea of sample rates and to what degree of accuracy we can quote the time stamps
+17:26 < sam_moore> I think I'll write some pin control code
+17:26 < sam_moore> I don't trust any of these custom libraries
+17:29 < jtanx> custom libraries?
+17:36 < sam_moore> Well they aren't really libraries
+17:36 < sam_moore> http://www.avrfreaks.net/wiki/index.php/Documentation:Linux/GPIO#Example_of_GPIO_access_from_within_a_C_program
+17:37 < sam_moore> Eg: That one has an fopen and fclose each time the value is changed
+17:38 < sam_moore> I could google until I find someone that has already written a C library, but chances are it will be slow or broken
+17:38 < sam_moore> Since I've worked out how to control the pins I may as well just write the C code to do it
+17:39 < jtanx> yep
+17:49 < sam_moore> I wonder if I can do this with a macro...
+18:30 < sam_moore> Ergh, screw that
+18:31 < sam_moore> Ok, I'm going to implement things like: GPIO/ADC/PWM_Export/Unexport to initialise or deinitialise all pins
+18:31 < jtanx> Ok
+18:31 < jtanx> too much effort with macros?
+18:31 < sam_moore> Yeah,
+18:32 < sam_moore> I was thinking of having something like "GPIOX_Set()" instead of GPIO_Set(int x)"
+18:32 < sam_moore> But that's probably not as nice as I thought it was
+18:32 < sam_moore> Anyway, there's an enum in the header file that contains the id of all pins used
+18:32 < sam_moore> The implementation defines some structs that wrap around the file descriptors
+18:33 < sam_moore> But to use the module you just give it an ID as defined in the enums
+18:33 < jtanx> Makes sense
+18:33 < jtanx> designing the gui is actually not too bad
+18:33 < sam_moore> That's good
+18:34 < jtanx> looks ok in ie8 too
+18:34 < sam_moore> Nice
+18:35 < jtanx> gotta go, dinner
+18:35 < sam_moore> Ok
+18:35 < sam_moore> Hmm, it would be nice if C had value checking on enums
+18:35 < sam_moore> You can define a function that takes an enum type as an argument
+18:36 < sam_moore> But people can still just pass any old integer
+18:36 < sam_moore> As far as I know
+18:36 < sam_moore> eg: typedef enum {FIRST=1, SECOND=10, THIRD=100} EnumType
+18:36 < sam_moore> void Foo(EnumType e);
+18:37 < sam_moore> If you go Foo(2) it won't complain
+18:38 < sam_moore> Annoying
+18:38 < sam_moore> That seems like something the compiler would be able to pick up
+19:31 < sam_moore> Ergh, I'm getting too obsessive compulsive with this pin thing
+19:35 < sam_moore> It's annoying because ADC, GPIO and PWM are treated completely differently
+19:35 < sam_moore> You write one thing and it enables *all* the ADCs
+19:35 < sam_moore> You have to enable each GPIO pin individually
+19:36 < sam_moore> And to enable PWM pins you give a string (not just an integer)
+19:37 < sam_moore> Also the location of the pin files is not guaranteed (though it probably doesn't change for a given system configuration)
+19:39 < sam_moore> Ah, I found a way to enable pwm with /sys/class/ instead of that cape manager thing
+19:39 < sam_moore> I think I'll use that, since at least it's consistent with the GPIO stuff
+19:41 < sam_moore> Ooh!
+19:41 < sam_moore> http://beagleboard-gsoc13.blogspot.com.au/2013/07/sampling-analogue-signals-using-adc-on.html
+19:41 < sam_moore> Provides a driver for continuously sampling with the ADC
+19:41 < sam_moore> Oh wait
+19:41 < sam_moore> Crap in a bucket
+19:42 < sam_moore> Because we're using those multiplexers we can't do that
+19:42 < sam_moore> We have to set the multiplexer before each sample
+19:42 < sam_moore> Oh well, never mind
+19:44 < sam_moore> I suppose we could write our own kernel module :S
+19:45 < sam_moore> I think I understand this enough to talk to Adam next time he tries to talk about sample rate
+19:46 < sam_moore> 1. It's not actually constant, but we can probably have it constant to within a few us
+19:46 < sam_moore> 2. To make it constant would require writing a kernel module
+19:47 < sam_moore> Unless electronics stops being stingy and gets one amplifier per channel :P
+20:22 < jtanx> hehehe
+20:22 < jtanx> next week's adrian though
+20:22 < sam_moore> Ah
+20:23 < jtanx> grilling time
+20:23 < sam_moore> He'll probably ask us the same things :P
+20:23 < jtanx> yeah
+20:23 < jtanx> but man, so much stuff to go through just to get some readings from a few pins
+20:24 < jtanx> so good job with that :P
+20:54 < sam_moore> Thanks
+22:45 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 24.0/20130910160258]"]
+--- Day changed Sun Sep 22 2013
+00:51 < sam_moore> Hell yes
+00:51 < sam_moore> PWM controlled through web browser
+00:51 < sam_moore> GPIO controlled through web browser
+01:19 < sam_moore> .... And ADC read through web browser
+01:19 < sam_moore> Blergh
+01:28 < sam_moore> I think I'll take the rest of today off from MCTX3420 :S
+11:53 < sam_moore> I've analysed the crap out of sampling rates for this ADC
+11:53 < sam_moore> At least as much as I can just using the timestamps according to gettimeofday
+11:54 < sam_moore> Contrary to my first email, reading the ADC is actually really slow. And also probably the greatest source of variation in sampling rate.
+11:56 < jtanx> wow
+11:56 < jtanx> only 100Hz?
+11:56 < sam_moore> Well it looks more like 1KHz on the oscilloscope, but there's a lot of variation, it has trouble getting a trigger
+11:57 < jtanx> the cpu datasheet rates it at 200kSPS
+11:57 < sam_moore> Hmm
+11:58 < sam_moore> Well judging by the control it is something about the ADC reading that makes it really slow
+11:58 < jtanx> That's annoyng
+11:58 < sam_moore> Yeah
+11:58 < sam_moore> Also annoying is that the ADC file is generally in a different place each time they're enabled
+11:59 < sam_moore> I ended up modifying the program to take the path to the ADC file as an argument
+11:59 < sam_moore> And making run.sh do the initialisation
+11:59 < sam_moore> I figured that was better than calling system()
+11:59 < jtanx> that makes sense
+11:59 < sam_moore> Yep, we might want to set other options that run.sh can pass to it anyway
+12:00 < sam_moore> Ok, I have to stop now, I'm spending way to much time on this
+12:00 < jtanx> Haha
+12:00 < sam_moore> It's getting to the point where I'm considering writing an ADC kernel module that doesn't suck :S
+12:01 < jtanx> :S let's hope it doesn't get to that stage
+20:12 < jtanx> the pressure regulator has a 1-5vdc analogue output
+20:12 < jtanx> is this considered one of the pressure sensors?
+20:14 < jtanx> or maybe it's just not used
+21:50 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 24.0/20130910160258]"]
+--- Day changed Mon Sep 23 2013
+08:51 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 24.0/20130910160258]"]
+20:55 -!- jtanx [
[email protected]] has quit ["ChatZilla 0.9.90.1 [Firefox 24.0/20130910160258]"]
+22:33 -!- Irssi: #mctxuwa_softdev: Total of 2 nicks [0 ops, 0 halfops, 0 voices, 2 normal]