Sunday, May 6, 2012

Presentation Practices and Final

Hi,

So, we spent the remaining in-class time doing presentation practices.  On the big day, we presented, and it went splendidly, all things considered.  The only thing I think could have been improved was that Nialls's speech could have been more polished.  Anyway, I'm glad it all worked out.

For posterity, here are my revised notes from the first presentation practice round:
From all the presentations, I have distilled the following ideas:
Presentation:
--Introduction (e.g. "Hi, my name is . . .")
--Introduce the concept immediately.  Explain motivation, what this product does.  Be concise!
--Address Marketing: Accessibility, Business/Pay model, Address Competition.  Mention that we're free and don't own any of the user's works.  Market toward the general population.  Mention professional web developers (there are two approaches: our product can be their starting point, or their replacement).
--Formality/Do not doubt yourself/Never admit to a bug/Do not cheer for basic functionality
--Make sure what you're doing in the presentation is actually visible.
--Watch the time
--Present tense!
--No one cares about technical details
--Practice everything. 
Demo:
--Practice setup (e.g. monitors, internet, etc.)
--Jump right into the demo right after the concept is introduced
--Show the "dragging" in the drag-and-drop website developer
--Make a website in five clicks, if you say that you can.
--Show theme-based version first.  Actually, probably don't need to show anything else.
--MUST practice.  Choreograph every single thing you do.  EVERYTHING.  DO IT.
--Video so that no problems possible?

Recommendation for overall structure:
1. Intro: "Hi my name is . . .", introduce motivation, introduce concept.
2. Begin a pre-planned, COMPLETELY choreographed demo.  Speaker continues, talking about things you can do with the website maker while the demo matches, doing everything in real-time behind him.
3. Demo continues, and a whole website is built while speaker addresses marketing, competition, etc.
4. The website, which has been being made throughout the talk, is exported and shown on a server.
5. Talk concludes, questions commence
It should be noted that I took more notes the second time, but they said essentially the same thing.  I didn't send them to the group either, so they're basically irrelevant.

Thanks,
Ian

Wednesday, April 25, 2012

Midterm Evaluations

Hi,

I got the midterm evaluation from Ackley back.  I'm honestly not too psyched.  I'd like to take this opportunity to reflect on the comments he provides.  He rates things on a 5 point scale.


Attendance, classes     |...xx|
Not entirely sure about this.  I've attended every class.  I'll take this not perfect score to mean my being late on a couple occasions for academic advisement--although that only happened the second half of this semester, so it doesn't really apply to the midterm evaluation . . .

Attendance, meetings    |....x|
Participation, classes  |..xx.|
Arguably I probably could have participated more in class.

Participation, meetings |..xx.|
Actually this should probably be lower.  Particularly of late, I let everyone else do the talking.  I suppose since this is the midterm evaluation, maybe it's fair.

Proposal draft          |x....|
Really?  When he gave the initial evaluation, it was a 1.5/5.  I guess he rounds down.  But honestly, it wasn't so bad.  It met the length requirements, and discussed everything it needed to.  It was rough around the edges, sure, but not that rough.  See also my overall reaction to the reviews.  I can only assume he's judging me against everyone else, and that mine was on the lowest end.

Proposal revision       |xx...|
Again, seriously?  I thought that, even if the original proposal was bad, the revised proposal was much much better.  It addressed literally everything that anyone complained about!  It was more clear, concise, and explained the idea better.  I worked hard on this, and it was most certainly not an incremental change.

Proposal reviewing      |..xx.|
This is about accurate I feel.  Maybe a solid 4 would be nice, but I *could* have reviewed more than two proposals, I guess . . . ?

Proposal pitch          |..x..|
What was wrong with it?  On Ackley's scale, this was "okay".  I memorized my presentation, practiced it extensively, and presented it next to flawlessly.  I even got some laughs from the audience, which very few other people did.  I even dressed for the occasion!

Proposal impact         |.x...|
Arguably correct.  It was a computer game, which isn't too important in the grand scheme of things.

Blogging                |...x.|
Also about accurate.  My blogging is pretty decent, if not stellar.

Interim self-evaluation |...x.|
Impact (Jan-Mar)        |..x..|
Overall (interim)       |.xx..|
                         --0++


Comments:


Proposal draft undercut itself right out of the gate.  Revision
improved in some aspects but doubled-down on the essential problem of the draft.  Pitch was mostly okay, given the content issues, if a bit weak on delivery.
As before, the delivery was pretty awesome I thought.  Maybe I just didn't project an aura of confidence or something.  I didn't stutter or pause or talk at my feet or any of the stuff you're not supposed to do, and I hit the time goal within 5 seconds, so Ackley, some clarification, perhaps?  I like to think of myself as a decent public speaker.  Ackley, if you have any suggestions, please let me know!

Also, on an accusatory note, I feel like my performance in the first part of the class is being judged based on Ackley's dislike of my proposal idea.  I understand that computer games aren't everyone's cup of tea, but that doesn't mean my proposal, reviews, and presentation of it necessarily sucked.  I'm trying to be tactful, but I'm not seeing any other way to explain these reviews.

Blogging has been a relative[ly] bright spot.  Quantity has held up overall, and posts are fairly often contentful, if fairly often blatantly self-serving as well.
I mostly agree with this.  I wouldn't call them self-serving.  As above, they're showing content, and if that content happens to paint me in a favorable light, so be it.  I'm not going to try to argue that I didn't focus on the best of my work, though, so maybe in some sense you could call it self-serving . . . ?

There is a lesson in continually getting the feeling that for some reason one's code is inherently better than code produced in the "real world".
Yeah.  I catch the double meaning; there are two lessons for me.  The first is: "I'm a pompous jerk.".   The second is: "Maybe I'm employable anyway."

Self-eval has lots of data.
Good.  I was hoping I was doing it right.

The utter absence of any interpersonal aspects in the self-eval areas for improvement is telling.
That's my bad.  I'm going to assume "intrapersonal" was intended (though both are applicable for this criticism).  I was under the impression I wasn't supposed to discuss my personal thoughts.  Though, it was a "self-eval".  I focused more on my achievements or lack thereof.  For the final self-eval, I will be sure to add this in.  It may not be telling in the way he thinks it though; the lack of interpersonal aspects is probably a manifestation of my psychological condition.  Really though, in the form given, it says to "focus on your achievements".  So, I don't know . . . ?

Overall: It really is good to be brashly self-confident; it really can be a valuable trait, but it also gets very old very fast -- when working with other people -- unless it is coupled with a willingness to listen and hear, to share credit generously where possible, and to take blame squarely and without weaseling when justified.
Sarcasm aside, I think this is valid criticism.  I am very confident in my abilities.  Part of that is justified by successful experience.  As far as my group is concerned, I certainly take my share of flak, and I like to think I do it honorably.  Let's face it--I'm a horrible liar, so I can't weasel out of anything by blaming it on other people, and I'm generally personable once you get past my outer, defensive shell of self-confidence.  Maybe that should go in my next self-evaluation.

Thanks,
Ian

Code Review

Hi,

So, I made a huge code review wherein I addressed all the issues I found within the project.  I looked for code-quality issues and bugs.  The process took at least 6 hours (which happened last Sunday, if I recall correctly; I really don't, nor do I want to).  The final code review was 61/2 pages and just about 2000 words long.

I sent the code review out to my group members, who had mixed reactions.  Luckily though, as a direct result of the review, a number of bugs were fixed.  Also, other people started reviewing each others' code, which I think can only be to the good.

Thanks,
Ian

Thursday, April 19, 2012

Eighth and Final Client Meeting, and Menu Serialization

Hi,

We had out eighth client meeting.  It went pretty well.  There was some consternation at a few things, but overall the demo succeeded.  Ackley also told us that this would be the final client meeting; that classes resume normally next week.

There's not much to say other than that the functionality is complete, and it's just bugfixes.

One issue popped up for the serializer/deserializer.  It appears that the menu objects are represented as Java objects (as they probably rightly should be).  The issue is that their internal representation is a canvas.  So, when we serialize the webpage, we serialize that canvas.  But, when we deserialize it, the canvas is deserialized, but the menu's object is not recreated.

I was not aware of this problem, and my group members only let me know about it around 10pm or so Tuesday night (the night before the client meeting).  I wondered why this was.

But, in any case, there wasn't time to fix it.  My solution was to deserialize normally, and then simply pick the canvas out, and add it to a new menu object.  However, Theo proposed treating the menu canvases as special primitively serializable types (like Integer and String).  I objected, but we eventually went with Theo's plan.  Ackley independently suggested the two solutions and made effectively the same value judgments we had about them.

So, we're working on the deserialization of menus.

Ian

Monday, April 16, 2012

Minor Serialization Update

Hi,

So, apparently, there was a minor problem with the deserializer in that it would set fields to null.  Apparently, SmartGWT seems to think that setting a String field to null is actually setting it to a value.  The practical upshot is that if you set a String field to null, SmartGWT will think that that null value is important in a way that it wouldn't be had you not done anything at all.

This resulted in some stupid problems, like hovering over elements giving null, and some warnings in the console. It fell on me to fix.

So, this morning, I wrote a Python script that converted the code to check for setting things to null.  The code worked perfectly the first time it ran.

Thanks,
Ian

Sunday, April 15, 2012

Cleaned up Code

Hi,

Well, I made a lot of the code prettier (i.e., it looks more simple).  I also renamed everything so that it's in standard Java conventions.  It irks me--because I'm a language purist--but the majority is always right, I suppose.  There's not really much else to say--as far as I know, all of my functionality is accomplished, and it's just bugfixes and code maintenance from here on out.

I'm pleased to boast that (nearly) all errors in my code were either due to a failing of GWT, or a use of the code for a purpose it was not designed.  I am not aware of any bugs in the current version of the source.  As far as I'm concerned, about the only thing left to do is to run through everything and make sure it looks nice.

I volunteered to examine everyone's code for bugs and other issues.  I will do this when I have more time and the code is more complete.  Group, if you need me to do anything in addition to this, let me know.

Thanks,
Ian

Wednesday, April 11, 2012

Seventh Client Meeting

Hi,

Well we had our seventh client meeting.  I think it went best out of all our meetings thus far.

As an aside, it should be mentioned that I changed a few things in light of slightly shifting requirements and fixed a few bugs since last time.  They took some time, but they're pretty uninteresting to discuss--and no design issues came up anyway.

One issue that came up was how I had changed a lot of fields from private to protected.  I explained that this is because those fields were used in an inner class, which technically doesn't have rights to private members.  However, the Java specification allows this (see 8.8.9), but it's technically discouraged.  If you set your compiler warnings high enough, you'll get it.

The other issue was the code review.  Ackley looked almost exclusively at Alex's code, so I felt kinda bad for him when Ackley ragged on it.

However, his complaints were pretty trivial, and were mostly stylistic (lowercase class names, e.g.).  I think this is overreacting and irrelevant.  If we want to use a different coding style, it should be up to us, really.  Coming from a strong C background, all my classes are lowercase; both myself and the rest of the group were therefore pleased that Ackley didn't happen to see any of my code.

In addition (and despite my denial of this), my code is apparently very dirty looking.  That's probably because it implements the most complex algorithms, is poorly commented, and makes non-standard use of syntax.  For next week, my primary concern will be to change my code so as to look simpler than it really is--because that's apparently what is expected of Java programmers.  Also, by fiat, I'm changing my code to use standard Java naming conventions.

Ian

Saturday, April 7, 2012

New Combination Logic

Hi,

Well, I got a lot done.

The group decided that we'd go with combining everything into a single hierarchy.  This has a few disadvantages, but as it happened it worked out well.

With respect to the new algorithm, it turns out that:
  1. The algorithm is (genuine surprise) simpler.  This is partly because:
  2. Positioning no longer needs to be changed explicitly.
  3. Technically, the first two levels of hierarchy are combined, instead of merely the first, just because of how SmartGWT implements it.
  4. In the rendered HTML, the page's canvas cannot have its own background color differing from the template's.  To do this, you will now need to make a div the size of the canvas and then fill it with color.  This is a direct consequence of merging the hierarchy, and is one of the disadvantages.  It may be necessary to try to merge the page's and the template's top level hierarchies, instead of just choosing one.  I may need some help with the HTML.
  5. The renderer will now attempt to figure out if something has a menu or not--no more passing in flags.  The algorithm checks for menus in the template AND in the page and will move them up (in the event of more than one menu being found (which shouldn't happen, I hope), the menus will have the same z-index and will be on top of each other).
  6. The algorithm *should* be safe on all valid data (e.g., you shouldn't be able to crash it by typing "<" in a string or something).
  7. The algorithm still fails if the canvas has overflown, because of a bug in SmartGWT.  The rest of the group should have been fixed already, but somehow, that's still happening.
  8. The spacing between the header, body, and footer, is not yet adjustable.  When I hear back from the group members on a desired API, I'll implement it.
  9. If you have more than 1000 different elements at the top level hierarchy, the Z-indexing won't move a menu past the first 999.  This makes the implementation cleaner, and it shouldn't be a problem.
As final points:
  • With the code review starting next Wednesday, we need to get the commenting and unit testing up to snuff.
  • If the template canvas overflows, it doesn't change the size of the index's canvas, so the two are different sizes.  This may be desirable for the footer, but I suspect it will ruin the alignment (which is relative, if you recall) if the overflow happens elsewhere.
  • I fixed all the warnings in the client package.

Thanks,
Ian

Wednesday, April 4, 2012

Sixth Client Meeting

Hi,

We had our sixth client meeting.  It went pretty well, actually, all things considered.  Importantly, the renderer worked exactly as expected.

There was some misunderstanding about the method of moving the template's menu to the front (a problem we had encountered that I had quickly written a fix for).  Ackley suggested that the template simply be moved in front.  The issue, as I (incoherently) attempted to explain, is that the template's parent could contain other things that shouldn't be in front.  I drew a pseudocode diagram as follows:

//template's div
<div ...stuff...
    -images
    -text
    -menu
//page's div
<div ...stuff...
    -images
    -text

My current algorithm is simply to take the menu out of the first div and add it into the second.

Ackley suggested simply changing the z-index of the menu instead.  Unfortunately, HTML doesn't work this way, so that wouldn't change anything.  Then, he suggested removing both div tags so as to put all the components into the same level.  I objected that this would raise positioning problems.  This is true, but as Ackley pointed out, those are workable and just because something is hard doesn't mean it's not the right solution.  What I was intending to say was more that those div tags contain important information (among which are positioning).  This includes backgrounds, images, etc. that the page really does need to override.

Now, again, that's not to say that this is impossible.  In subsequent discussions with Theo, we established that removing the top level hierarchy is indeed possible functionally--it's just not as simple as Ackley thought.  You can't just delete the top level hierarchy.  SmartGWT adds it in for a reason, and it's more subtle than just setting up styles and positioning.  Like I said, it's not impossible to remove, but it's not something that necessarily improves the result in terms of sensibility.

This brings up another point.  The current algorithm works and could be trivially generalized for anything else we want to move.  However, it maintains the current hierarchy.  We could also do (a version of) what Ackley suggests and remove the hierarchy (or at least combine it).  Doing anything else, Ackley calls "a hack"--but I definitely think that's too narrow of a perspective.  I'm not sure what the "right" solution is--and the current algorithm has intrinsic merit--you're separating the template's stuff from the page's stuff--transferring to the page everything that really should be there in the first place, so it makes sense.

I'll admit that don't know what the "correct" algorithm is, in terms of sensibility.  But, I will be discussing this with the group, and I will ensure that we have that algorithm for next week's meeting.

Ian

Sunday, April 1, 2012

Renderer Fixes and Positioning


Hi,

I looked through the renderer. The reason it was crashing was because someone passed in an incorrect URL as its final parameter. This caused an out-of-bounds error. I refactored that code so that that's outside of the renderer itself, so now it's more clear what's going on. The baseurl is a directory of the .CSS files. I don't know what that should be, because whatever it was before got changed.

Secondly, this fix automatically makes the renderer work again. However, it would still output nothing if nothing substantial is passed into it. This can happen, for example, after the renderer has already been called once (the canvases seem to get screwed up still). I can't say that was *entirely* not my fault, because half of the problem had to do with an undocumented bug in GWT, which in turn caused a problem in my code. Anyway, I fixed it, and the output corresponds to the input in all test cases I've tried.

However, someone changed the positioning to relative positioning, so the positioning-hack for superimposing the template and pages doesn't work anymore. I understood this was going to happen eventually, so it's not unexpected. I then figured out an algorithm to get relative positioning working. It's pretty simple, and I kept the old algorithm just in case. It correctly positions both the headers and footers. There's still a problem, though. The height of the template's footer is 34px. The height of other pages' footers is the default, 100px. I suspect this to be a bug somewhere else. Once this is fixed, I'll finish up the position algorithm by getting the footers' positioning working.

Thanks,
Ian

P.S.: The whole thing about the (de)serializer and renderer changing the editor afterwards? It turns out that new canvases don't go away unless you explicitly kill them with .destroy()--garbage collection evidently doesn't do anything. This was news to me, anyway. I haven't observed the problem with the (de)serializer, so perhaps Theo figured this out too. Anyway, I've also added fixes to the renderer to solve that as well.

Wednesday, March 28, 2012

Fifth Client Meeting

Hi,

Well, actually, the meeting went better than I had previously expected.  However, there was a LOT that could have gone better.

Notably, the serializer and deserializer worked pretty well, with issues.

Somehow, the parser didn't work at all.  This is problematic, particularly since I didn't change the code since last night, when it did.  This means that someone changed the code so that it's passing in garbage to the parser, so it either gives a warning and crashes, or produces erroneous output.  I was very disappointed in this.  It should have worked perfectly, and I was looking forward to showing it off.

That said, at least internally, we have a proof of concept.  As a group, we know all of the technology in play will be fundamentally valid.  It's just a matter of getting it to work properly.

Let's do this.

Ian

Tuesday, March 27, 2012

Updates

Hi,


So, the serializer and renderer are not completed.  Those were my areas of responsibility.  While I think that the fact that they are not yet completed is a product of factors that aren't all of my doing, I personally said I would get these done, so it's ultimately my fault.  If worst comes to worst, I'll take the blame for it.

Now.  Not all is lost, of course.  The deserializer is coming along.  That will need to be finished.  The serializer *should* be done already.

As far as rendering, I am pleased to announce that the key link between GWT's retarded API and my HTML parser has been discovered (albeit through some particularly nasty recursion)!  As a result, the renderer can now take a canvas and spit out HTML that corresponds to it.  It's not perfect, but I think it will satisfy for tomorrow (next client meeting).  As proof, see this HTML file from the generator: http://cs.unm.edu/~imallett/other/460/Untitled-2.html.

Thanks,
Ian

Monday, March 26, 2012

Serializer and Converter

Hi,

I finished the basics of the serializer.  It serializes all children.  A couple of fields are still incorrect, but they're not as important.  I subsequently released control of the module to the group to improve.

Now, as far as the HTML converter is going, it's not.  Don't get me wrong; the parser, formatter, and synthesizer described here: http://iancs460.blogspot.com/2012/03/revised-parser.html are all working perfectly!  The renderer just isn't.  Right now, all that's happening is the renderer is calling "canvas.getElement().getInnerHTML()".  This produces the final output:

<html>
  <head>
    <title>
      [PAGE TITLE HERE]
    </title>
  </head>
  <body>
    <div id="isc_7P" eventproxy="isc_WebPage_0" class="normal" style position: relative; z-index: 201872; visibility: hidden; background-image: url(http://1.bp.blogspot.com/-h-aleanaggc/tcbtz8mgaai/aaaaaaaaalq/hofsrycnkle/s320/checker.png); overflow-x: visible; overflow-y: visible; left: 5px; top: 5px; width: 1373px; height: 936px; background-repeat: repeat repeat; onscroll return isc_webpage_0.$nd()>
      <div id="isc_7Q" eventproxy="isc_WebPage_0" style="POSITION:relative;VISIBILITY:inherit;Z-INDEX:201872;CURSOR:default;">
        &nbsp;
        <div id="isc_7R" eventproxy="isc_Canvas_17" class="normal" style="POSITION:absolute;LEFT:0px;TOP:0px;WIDTH:1373px;HEIGHT:902px;Z-INDEX:201890;OVERFLOW:visible;" onscroll return isc_canvas_17.$nd()>
          <div id="isc_7S" eventproxy="isc_Canvas_17" style="POSITION:relative;VISIBILITY:inherit;Z-INDEX:201890;CURSOR:default;">
            &nbsp;
          </div>
        </div>
        <div id="isc_7T" eventproxy="isc_Canvas_16" class="normal" style position: absolute; z-index: 201908; overflow-x: visible; overflow-y: visible; left: 0px; top: 0px; width: 1373px; height: 936px; onscroll return isc_canvas_16.$nd()>
          <div id="isc_7U" eventproxy="isc_Canvas_16" style="POSITION:relative;VISIBILITY:inherit;Z-INDEX:201908;CURSOR:default;">
            &nbsp;
            <div id="isc_7V" eventproxy="isc_WebPageCanvas_3" class="header" style position: relative; z-index: 201926; overflow-x: hidden; overflow-y: hidden; left: 0px; top: 0px; width: 1373px; height: 135px; onscroll return isc_webpagecanvas_3.$nd()>
              <div id="isc_7W" eventproxy="isc_WebPageCanvas_3" style="POSITION:relative;VISIBILITY:inherit;Z-INDEX:201926;CURSOR:default;">
                &nbsp;
              </div>
            </div>
            <div id="isc_7X" eventproxy="isc_WebPageCanvas_4" class="body" style position: relative; z-index: 201944; overflow-x: hidden; overflow-y: hidden; left: 0px; top: 10px; width: 1373px; height: 722px; onscroll return isc_webpagecanvas_4.$nd()>
              <div id="isc_7Y" eventproxy="isc_WebPageCanvas_4" style="POSITION:relative;VISIBILITY:inherit;Z-INDEX:201944;CURSOR:default;">
                &nbsp;
              </div>
            </div>
            <div id="isc_7Z" eventproxy="isc_WebPageCanvas_5" class="footer" style position: relative; z-index: 201962; overflow-x: hidden; overflow-y: hidden; left: 0px; top: 20px; width: 1373px; height: 45px; onscroll return isc_webpagecanvas_5.$nd()>
              <div id="isc_80" eventproxy="isc_WebPageCanvas_5" style="POSITION:relative;VISIBILITY:inherit;Z-INDEX:201962;CURSOR:default;">
                &nbsp;
              </div>
            </div>
          </div>
        </div>
      </div>
    </div>
  </body>
</html>

This looks pretty good, but it doesn't render anything else.  I suspect this may be due to it's not calling the same on its children?  In which case, the fix might be simple.  If not . . .

See also: http://stackoverflow.com/questions/9855735/get-smartgwt-canvas-as-html

Ian

Sunday, March 25, 2012

Progress

Hi,

Well, we're making progress.  On the serializer, the correspondence between the input and the output is nearly one-to-one in the ways that matter.  The problems are the "left" data field is not being set, and the children are not being serialized.

I don't know why the first is the way it is, but the second is due to me simply not implementing it yet because it's hard.  However, I'm about to go do that now.  Hopefully by tonight or tomorrow evening the "serializer"/"deserializer" will do everything it needs.

I'd also like to note that Nialls has done a nice job improving our home page.  There are now joke testimonials and a cleaner layout.  I think I might like to work on some web design soon, seeing as the algorithms section of this project is nearly wrapped up.

Ian

"Serializer" Hacks

Hi,

Well, GWT is broken and buggy as ever.  For instance, the methods .getEdgeSize() and .getExtraSpace() both fail on a newly created canvas with a NullPointerException.  Turns out that typing too fast (e.g., my auto log-on script) can crash GWT, as well as certain properties.  GWT at least catches errors and gives you nice feedback if you do some things (like call some methods after construction).  Though, I'm not sure how you call a message before construction?  And yes, GWT still crashes every now and again, which is . . . disconcerting.

The API is inconsistent between the getters and setters, which made matching everything up a huge ordeal.  Not to mention that the API has (literally!) about 200 getters and setters.  Not all of them are necessary, probably--but how am I to know?  The code is horribly messy (notably because Java doesn't have function pointers, nor macros) too.

It does kinda remind me, though, of how slow the whole thing is.  GWT takes forever to start, then the website has to load up.  Then you have to click login, wait for the window to come up, enter your username/password, click send, wait for that to go through, open a new project, and *then* you can click "save", and actually run a single test!  Our code isn't exactly slow per-se; the process to do something is just soooo long!

In any case, after much complaining and whatnot, the serializer/deserializer kinda works.  A canvas can be taken in, serialized, and then deserialized.  I don't know if the correspondence is one-to-one in the important ways (except for what I know isn't, like recursively evaluating children), but it's a strong start.

Tomorrow (today) I'll finish it up.  I DO have other classes too, so I'll need to work on that as well.  It's just about 1:45AM, so, knocking off for the night.  See you bright and early.

Ian

Saturday, March 24, 2012

Serialization Update 2

Hi,

Well, the serialization fiasco has mostly been figured out.

It turns out that yes, my serializer does work.  Moreover, it works perfectly, and everything would be good . . . except that the deserializer does not.  As it happens, Java's deserialization is, put simply, broken.  There's nothing for it.  In real life (C++) I'd use a copy constructor, and that would be the end of it, but Java apparently can't handle that.

Unfortunately, the deserializer doesn't, and can't work, unless the GWT designers had thought of it, which they didn't.  We'd be stuck, except Joe Collard put into words what I was coming to the realization was necessary (on same post by Nialls mentioned previously, here): "see if you can write a wrapper that contains all of the things you will want saved".

Essentially, the idea boils down to: write your own serializer.  It's bloody messy, but Java's (de)serialization is broken and reflection isn't supported by GWT.  So, we have to go with it.

No private/protected fields can, of course, be serialized.  Hopefully this shouldn't be an issue.  It appears to have worked for Joe's group, so hopefully it will work for ours.

Ian

Serialization Update

Hi,


So, when running the serializer, GWT will produce errors:

[ERROR] [web_final] - Errors in 'file:/C:/dev/Java/CS%20460%20Project/Web_Final/src/cs460/webdnd/client/utilities/serialization/serializer.java'
 [ERROR] [web_final] - Line 24: No source code is available for type java.io.ByteArrayOutputStream; did you forget to inherit a required module?
 [ERROR] [web_final] - Line 25: No source code is available for type java.io.ObjectOutputStream; did you forget to inherit a required module?
 [ERROR] [web_final] - Line 39: No source code is available for type java.io.InputStream; did you forget to inherit a required module?
 [ERROR] [web_final] - Line 39: No source code is available for type java.io.ByteArrayInputStream; did you forget to inherit a required module?
 [ERROR] [web_final] - Line 40: No source code is available for type java.io.ObjectInputStream; did you forget to inherit a required module?
 [ERROR] [web_final] - Line 47: No source code is available for type java.lang.ClassNotFoundException; did you forget to inherit a required module?


In response to this, Nialls took over serialization/deserialization using code he found online.  This code produces tons of errors, as he writes here: http://niallsc.blogspot.com/2012/03/canvas-serialization-status-failed.html.


However, this misses something--my serializer worked.  It produces this output (if you add a print statement):

Now, this looks to me like taking a canvas and serializing it (and I can get a canvas back, deserializing it).  I'm not sure what the problem is with Nialls's code, but mine seemed to work.

I feel somewhat accused, but I submit the above as evidence that the code I wrote is working properly.  There were limited tests with more complex canvases (nested, with content, etc.) that were conducted some time ago, successfully.  It is possible that there is some new problem (like maybe a certain component can't be serialized) which can be solved in the same way as the canvas's serialization was.

Now, I don't know what to say.  Maybe there's something subtle going on that I'm missing.  Maybe I don't know what I'm talking about.  Maybe there's a fundamental flaw in this that really will somehow prevent this from working.  I welcome explanations.

Regardless, I think it's a good idea to meet Sunday as Nialls suggested to work this out.  For the good of the project, it needs to be solved sooner rather than later--and I had already thought it was solved.

Thanks,
Ian

Serializer/Deserializer

Hi,

So, since last time, I have added a new class "CanvasSerializable", which is a direct subclass of "Canvas", except it adds support for serialization.  If every canvas is serializable, then making the serializer work properly is much easier, and allows for better design.

The changes affected a lot of the project, so I changed every single reference to Canvas to CanvasSerializable, and then converted things back that just flat out couldn't be (e.g., GWT callbacks return Canvas[], so I can't upcast them to CanvasSerializable).

This seems to have had an effect already.  The serializer appears to be working properly.  There's still some problems on the server-side (I think) so serializing doesn't quite work.  Because there's no serialized data, naturally, the deserializer doesn't work either.

Hopefully, over the next few posts, the serializer module may be completed!

Ian

RSA Cryptography

Hi,

I wrote a little cryptography module.  It implemented a (theoretically) fully secure RSA cryptosystem.

This turned out to be harder than anticipated.  In addition to jogging my memory of RSA, implementing it in Java is a nightmare (exercise of actual real programming).

It turns out that there are two different APIs for this.  The one most directly implemented by Java has a bug that prevents it from working except when writing to a file.  The second is also too buggy to use.

So, I decided to write my own implementation.  It turns out that there are a number of (buggy) Java implementations.  These fail in a number of cases, (usually when a message is too long).  Obviously, real RSA doesn't crash.  I ended up basing my solution on an existing RSA implementation and then rewriting half of it to actually work.

In the process, I also tried compressing the RSA output stream.  It turns out that the Java "Inflate" class has a bug that causes it to hang on the general kind of data that can be produced by compressing an RSA stream.

I can't really do anything about that, because if I were to try to fix all the bugs in the JRE, I should at least get paid--and anyway, at this time, there are more pressing matters, such as serialization, updates concerning which should appear on the next post.

Ian

Thursday, March 22, 2012

Compression

Hi,

Well, I took a little break from doing parsing and serialization whilst I wait for others to get set up to use the serializer/deserializer.

Some, especially Alex, expressed concern that serialized data or the generated HTML would be very large and difficult to send.  So, I wrote a compression wrapper around Java's Inflate/Deflate APIs.  It turns out that Java's String.getBytes(...) method is broken, which made this difficult.  After some false starts, I finally just gave up trying to work around their bugs and just wrote out what the methods were supposed to do myself.

The algorithm is the standard ZLIB compression algorithm, which should be able to achieve very good compression rates.

I may try writing an encryption algorithm as well.

Ian

Regular Expression for Tag Processing

Hi,

Last time I mentioned that regular expressions might be useful for breaking a tag into elements.  My regular expression is:
([^\s\"=]+\s*=\s*([^\s\"=]+|\"[^\s^\"]+\"))|([^\s\"=]+)
[Edit: Updating for single quotes and other improvements: ([^\s\"\'=]+\s*=\s*([^\s\"\'=]+|[\"\'][^\s\"\']+[\"\']))|([^\s\"\'=]+)]

Notice that it is awesome.  I tested it on the input data:
div class=hijkjh width=100 height = 6 src=   "hello.png" why=bec6=ause src="hello.png" src="hello.png?=yo?" src="hello.png" src="hello.png" src="hello.png" sup

. . . which is deliberately malformed in some cases.  You can test the regex at http://regexpal.com/.  Notice that the regex matches valid elements.  All I have to do to determine if a tag is well formed is check whether anything other than whitespace is unmatched.

I have merged these changes into the converter itself, so the new tag system should work much better.

Ian

Revised Parser

Hi,

I revised the parser.  It can parse much better.  I fixed a few problems, cleaned up the code some, and in the process actually was able to make the necessary algorithms shorter.

I ended up not using regular expressions, because they were basically overkill.  All that was really necessary was to split the input stream into tags and content, instead of merely tags.  I kept most of the rest essentially the same (although I refactored the main algorithm into separate functions for cleanliness).

So, progress was made.  However, a new issue has arisen.  The parser can't properly parse something like <img src="imgs/srv?=cheese.png"> because of the equals in the quoted string.  In practice, the parser will spit out a warning that the HTML is malformed, and just output the tag as-is.  So, the parser is not fundamentally broken; it's just that it is properly formed and the parser just doesn't understand it.

When I was creating the parser, it never occurred to me that such situations could arise, so I deliberately attempt to have the parser look for equals.  In this case, the parser finds two equals inside what it considers one element: (e.g., for <div class="hello=freedom">, it thinks it has gotten a <div> tag with a single element of the form A=B=C, which is malformed).  The quotes don't matter.

To fix this, I may use regular expressions.  Though, as trends are going now, I'll probably think I will and then end up not.

Finally, from here-on out, the parser is now being called a converter.  The converter comprises the renderer, which converts GWT into HTML, a parser, which breaks that HTML up into lexical chunks, a formatter, which cleans up the HTML and formats everything, and a synthesizer, which tidies up and combines everything.

Ian

Wednesday, March 21, 2012

Fourth Client Meeting, Updates, and Serialization/Deserialization

Hi,

As catchup from last time:
-With the parser complete, serialization and deserialization of raw Canvas-es is next.  This is necessary for saving and loading.
-The parser may not actually be complete; more formatting may be required and there are other issues.
-Had a client meeting.  It went poorly.

The serialization and deserialization is pretty straightforward so far.  It uses a slightly modified version of the standard Java serialization routines.  It's not fully functional (it doesn't reconstruct everything properly yet), but it's in-place and a good start.  Nialls will pass it the actual canvases, and then the real fun will begin: getting it working with "real" data.

The parser is annoying me.  It turns out I actually wrote pure XML parser instead of an HTML parser.  So I'll need to make some *significant* changes.  I'm thinking of resurrecting regular expressions.  I'm dedicating this afternoon/evening to doing this.


We also had our fourth client meeting.  Honestly, I think it went the worst out of all client meetings thus far.

I think a lot of this had to do with the group not being very productive over Spring Break.  I freely admit that I definitely slacked, and I feel bad about it, particularly since my modules were least integrated out of everyone's.

I feel like I didn't get credit for making the serializer/deserializer from Ackley.  Yes, it wasn't fully integrated, but *it worked*.  I also feel like Spring Break ought to be a break.  I know that the reality of it is that work happens over Spring Break and that I'm expected to do things.  But I still think it's problematic to call it a "break" if it really isn't to our society.  Same applies to Christmas Break, for year-long classes.  In the extreme, Summer, for some summer assignments.

So, for next time, the serializer/deserializer, and a revised parser.

Ian

Friday, March 16, 2012

Parser Complete

Hey,

I'm pleased to announce the alpha version of the parser!  The parser can take any GWT canvas and convert it into HTML, the formatting of which can be adjusted to as desired.  Additionally, it can take HTML and make inferences to make it valid HTML (this is important because GWT outputs non-standards-compliant HTML).

Currently, the parser can be customized to output/not output newlines and pipe characters (for viewing alignment) and whether tags are capitalized/not.  I didn't really know what other options to add, but it should be trivial to add them.

The thing is pretty robust, in that it will take any HTML you could possibly throw at it.  It will even detect some problems (e.g., "<div class=a=b>") and let you know about them.  I can't yet give it my (imaginary) gold seal of approval, but it's definitely solid enough for alpha.

Anyway, this concludes my original goal of getting the parser working.  The parser is fully integrated into Web_Final, and is ready for use.  All changes have been committed.

TODO:
1. parser does not attempt to find redundant or useless code (e.g., "<div></div>" . . . I (think?) is useless).
2. Do we, or do we not want HTML to GWT conversion?  Depending on the answer to 1., this may be as simple as removing the useless code and then feeding the generated HTML back into a canvas object.  But I honestly haven't looked into it.

Thanks,
Ian

Sunday, March 11, 2012

Project Mid-Semester Self-Evaluation

Hi,

I wrote my self-evaluation.

I honestly only see a marginal purpose in it: I understand the rationale for a self-evaluation, but I somewhat object to the notion of it being the programmer doing it (I was under the impression that evaluation was a function of a manager. Am I doing a manager's work?). Putting it in writing is also annoying, although I do recognize the force of having something in that medium to refer back to later as a concrete reference.

In any case I gave it a go, and, as near as I can tell, executed the assignment well. If the client has any problems with what I did, I'd appreciate knowing about it in advance so that I may fix them (although I understand if that's unreasonable).

Thanks,
Ian

Scratch Regular Expressions

Hi,

We had another group meeting this Sunday, and we got everyone on the same page for the project. In particular, we fixed a lot of issues pertaining to project setup that were preventing most people from getting anything working. Apart from a delay due to DST, the meeting went well.

I have been working on the regular expression matcher--or, more precisely, I was. I decided that regular expressions are messy and overkill for the application (purpose being merely to reformat the GWT output).

Instead, the new plan is to use a simple algorithm (string search) to find and format tags. This is simpler and cleaner. In addition, it has the advantage of being strongly linear time (which is only the best case for regular expressions). I could throw in some flak about Java not supporting pointer arithmetic, but at this point, I'm just glad that I'm writing algorithms :-)

Additionally, there will need to be a formatter API for the parser. This is implemented as a simple class that can (optionally) be passed to .generate(...). The class is analogous to a "struct", in that the point is to set various data fields, with others being set to defaults, and then pass it to a function.

I'm not entirely satisfied with this implementation, because the "struct" needs a public method ".apply()", which applies the struct's data values to an input string. It's paradoxical because this method must be public so that the parser can call it, but only the parser needs it. So, I'm violating encapsulation. Yet, I can't just move the .apply() method to be a private method of the parser because that violates encapsulation (the method really should be a method of the "struct"). In C++, I'd make parser a friend class, but in Java . . .

Final notes:
-I'm creating a new project wherein I can test my parser separate from the other modules of the program. Normally, I wouldn't care, but GWT is too unreliable and slow to fit my mandate of obscene amounts of runtime testing. It's a code quality thing.
-I'm working on the project mid semester self-evaluation (http://cs.unm.edu/~Ackley/460/selfEvaluation.html). I'm . . . not psyched.

Ian

Wednesday, March 7, 2012

Third Client Meeting

Hi,

Well, the third client meeting has come and gone. Honestly, once again, I think it went pretty well. I think we've got the presentation format down fairly solidly: a brief summary of last week's accomplishments, and focus for future work in the coming time ahead, followed by a demo with specific questions deferred to those who know best.

As for me, the parser is functionally complete. It parses all GWT canvases into HTML flawlessly, as far as I can tell. The only drawback is that the generated HTML is horrible. I decided to forgo recursive descent parsing in favor of letting GWT do the heavy lifting. Ultimately, this should (maybe? I don't know, given GWT's robustness record) be more stable and resilient to improvements group members add that take advantage of more features of GWT. However, in classic GWT bloatedness, the HTML has only limited customization options--and nothing is able to resolve redundant and useless code (e.g. "<div></div>").

My solution? Write a regular expression meta parser/converter for the HTML parser's output! This should at least be interesting, and will have the benefit that we will have beautiful control over the generated HTML. In contrast to GWT, I will ensure that there is a clean API for this.

By next time, I hope to have at least a start on doing this.

Other notes on the meeting:
-User stories appear to be wanted.
-I speculated that perhaps we could serialize and deserialize Canvases to save them and restore them. This may not be possible, although I don't know why you can't just subclass Canvas and add "implements Serializable". Maybe some internal thing?

Ian

Monday, March 5, 2012

Mid-Parser and the Importance of Software Engineering

Hi,

The parser proceeds well. I'm currently refactoring the code to be more modular and robust, as well as cleaner. I have my own subpackage in the solution, and I'm going to fill it with as many files as I need to make the architecture pretty.

In terms of functionality, I wasn't able to accomplish anything over the weekend. If you must know, I instead wrote a GPU raytracer for rendering 3D fractal geometry, complete with distance estimators, Phong shading, and frustum clipping. Woe that this isn't a graphics project.

However, I had planned on doing that; the parser will still be advanced in time for the next client meeting on Wednesday. Positioning will be in place. In fact, I anticipate, the way things are going, for the parser to be just about completed, with minor functionality to be added as new features are made accessible by the GUI.

Meanwhile, GWT is being snarky again. Actually, strictly speaking, not "snarky". That word is closer to "sassy" or "cheeky", and at least implies some level of sophisticated defiance. No, GWT is being outright stupid. My primary complaint right now (because I'm not so directly working with it) is that it's not robust. It takes like ten seconds to start up, and when it does, half the time there's some internal error. I've found that restarting Eclipse usually helps. But, I am frustrated that something that's supposedly super-awesome can't even get out the starting gate when you try to use it. And, I'm immediately suspicious of something that runs differently each time. It brings to mind thread contention issues and broken dependencies. And I'd be worried about that except GWT is necessarily single-threaded. . . . which still leaves the question: why can't GWT work perfectly every time? Is that too much to ask?

It brings up an interesting point about software engineering. I consider myself much more a computer scientist and programmer, instead of a software engineer. The difference is subtle, but I talk about it a bit here.

Nevertheless, I continually get the feeling that for some reason my code is inherently better than code produced in the "real world". It is endlessly annoying to have to work around someone else's bug in a library, or to write wrappers around broken APIs so that they're usable. Yet somehow, for some reason, my code always manages to be fast, flexible, and, above all, robust. I frequently (weekly or so) receive emails from satisfied users of my libraries praising their "beautiful API[s]" and robustness. In one coder's words:
". . . the API is cleasn[sic], crisp[,] and compact--yet still powerful. Your code is astoundingly bug-free and my only problem[s] come from me being completely retarded. Using your code is like breathing freely."
I myself hardly ever have the opportunity to send such mails. I typically send bug reports.

The point is not to demonstrate that I'm a better person or something--the issue is one of design. Better-designed code tends to work better. Do I just pay more attention to that than other people? These questions go through my head rather more often than I'd normally care to admit, and they are of particular relevance as I rework the parser. Group members: you may expect robust, powerful, and well-designed code.

But it all brings me back to GWT. I'm not going to speculate on why it can't reliably start, but it suffices to say that it makes debugging extremely annoying, so long as "extremely" is VERY strongly emphasized. I think I may start a separate project, and run the parser on GWT objects that have been saved out and serialized. I wager it would be faster--though, the way things are going, I wouldn't be (too) surprised if GWT has some meaningless constraint that prevents you from doing this.

I also found a curious little method Canvas.getInnerHTML(), which appears to return a whole lot of nonsense. It looks like some of it might be useful, but it's clearly no substitute for real parsing.

Which reminds me, back to that design thing: one good design principle is to avoid name collisions. I solve this with namespaces, and by naming my functions and classes intelligently. At the very least, prefixing with some easy to recognize identifier.

In Java, the evidently accepted solution is to use different capitalization, because apparently that's so much cleaner. On right, what Eclipse has to say about the unrecognized symbol "HTML".

Terse and uniform APIs? Mmmmm no.


So, project-wise and in sum: the parser will be next to done by Wednesday. Furthermore, it will live up to my standards of design. GWT is annoying. We'll see how it plays out.

Ian

Friday, March 2, 2012

Second Client Meeting

Hi,

Well, in contrast to last time, our presentation this time went very well. We were much better organized and we covered the material linearly and logically.

However, I also think that there were some issues. The most noticeable was our use of time. We were cut somewhat short on time. I was under the impression that we had 15 minutes. Apparently, we have 10.

In any case, Ackley apparently doesn't care about PowerPoints. That's fine; we can work with that.

Solution: future presentations will be all demo, and as we show various sections, group members will discuss their contributions. This should be more entertaining and make better use of time.

Ian

Tuesday, February 28, 2012

First Parser!

Hi,

Well, I've been horribly busy, but, I have successfully implemented a very basic GWT parser! It doesn't yet do all that it will eventually need to do, but it definitely does a lot of it, including outputting (some) of the header and a fair bit of the body.

I also was successful in integrating the parser into the main project, in time for the presentation on the 29th.

Running the parser on a "blank" canvas gives the following as of now:
<html>

<head>
<title>[PAGE TITLE HERE]</title>
</head>

<body>
<div>
<!-- CANVAS "Canvas{ID: "isc_Canvas_0",
position: "absolute",
className: "normal",
width: 100,
height: 100,
children: Array[2],
cacheOffsetCoords: true,
zIndex: 200036,
}" GOES HERE -->
</div>
</body>

</html>
This should be a valid webpage, even if it's not very interesting . . . yet.

I also created the PowerPoint for our presentation tomorrow. I made the basic structure and did a lot of the editing.

Ian

Monday, February 27, 2012

Changing Roles

Hi,

Honestly, I'm a failure at GWT. It drives me insane. I don't know what it is--the poor design, the convoluted and broken API, or the lack of documentation so that I can work around it. Maybe I just haven't learned to love it yet, and I'm giving it undeserved flak. Maybe everything somehow makes sense. I don't know. But I can't stand it.

That's why, henceforth, I am distancing myself as much as possible from GWT as possible. I have changed my role in the group. I will be writing code to parse GWT objects into HTML. I can do this; it's more low-level and less tied to GWT's API than making the objects in the first place. I also have experience with HTML.

While Nialls works on the GUI, I'll be taking the canvas objects he creates and turning them into standard HTML. For now, the HTML will be dirty. Among other things, it will make use of tables for alignment.

The design challenges here will not be trivial. Data conversion is difficult to execute in a clean, crisp way. In addition, there's the meta challenge of getting the HTML itself to have a decent design, even if it is using tables. I honestly don't see what the big deal is, just so long as the HTML isn't poorly written.

Additionally, after the fiasco of last week's presentation, I have tasked myself with creating the group's presentations. Presentations will begin with one person (me) describing at a high level the group's accomplishments, the architecture, where we're going, what we've been working on, and any bureaucracies that need to be addressed. From there, the individual group members will give more in-depth discussions of what they are doing and how they are doing it. After this will come questions.

So, I will be doing GWT to HTML conversion and will be handling organizing the presentations. For next time, there will hopefully be a spike of minimal GWT to HTML parsing, and our next presentation.

Ian

Thursday, February 23, 2012

First Client Meeting

Hi,

Well, we had our first client meeting today. I can't say it went . . . spectacularly.

I think a lot of the problem was that we talked about what we did individually, instead of as a unified whole.

Well, we know better now. In the future, presentations will be more like presentations instead of journal dumps. We're working a lot on group communication, so hopefully, we'll be more coherent.

Additionally, I think perhaps it would be most wise to have one person summarize the group's proceedings, and then individual questions may be deferred to those who know best.

Ian

Tuesday, February 21, 2012

Menu Spike Postmortem

Hi,

Well . . .

I've played around with it some more, and I'm disliking GWT more than I'd originally thought. It seems very powerful, but there's basically nothing coherently organized about how to use it. I find stuff on the GWT site like "GWT is being used by tens of thousands of projects around the world.". Uh huh.

In any case, the good news is that I've successfully completed a spike. It's ugly and bad, but it works. It may be found in the SVN repository.

The spike is (VERY heavily) based off of one of the other spikes (presumably Nialls's). I cleaned up a lot of things, in the process trying to get some semblance of understanding. In the end, I removed a lot of the hardcoded stuff, and tried to abstract it. It still does basically the same thing.

The primary issue this raised, however, is how the general structure will be set up. The main concern I have is the use of canvases versus the use of panels. Everything I saw said that you should use panels. However, in the spike I was working from, canvases were used.

From what I dimly understand, panels would allow automatic positioning and sizing, which is something this project will desperately need. On the other hand, again, as I dimly understand it, canvases allow drag and drop functionality. And I have to say, it looks really good. Unfortunately, I wasn't able to find anything useful about this schism in functionality and how to resolve it.

The other, somewhat tangential issue, is copying of objects. When dragging, a new object is created. That's fantastic, but it does mean that the old object's settings need to be copied over. The spikes currently just hardcode it, which is dirty. And the GWT designers, in their infinite wisdom, didn't provide some adequate way of doing this--or, at least, making it obvious. I hacked up a deep copy using what essentially amounts to serialization, but the software complained that it didn't know what the classes I used are. So, GWT-created pages don't support all of Java? Okay, that would have been nice to know. In any case, the upshot is that copying might be an issue.

Ian

Menu Spike

Hi,

Well, I've got a start. I've decided that I'm going to reverse engineer some of the previous spikes. I reconfigured Eclipse to be more strict, so now a number of problems have been caught earlier.

GWT is ugly, but at least using it is falling-down simple. So, I don't anticipate too much of an issue. I think that it will definitely be possible to get something at least working well by Wednesday, even though I'm only (really) starting the spike today, Tuesday.

We shall see.

Ian

Monday, February 20, 2012

GUI Design

Hi,

So I'm doing a lot of the GUI design, mostly because this is the only thing I can really latch onto in the project.

I met with Nialls, and we agreed on the general structure of the GUI. I will be writing a spike that demonstrates menu creation using GWT. This is pretty much the trial by fire for me with GWT, so we'll see how it goes.

I have given myself until Wednesday morning to write something rudimentary, because that is when our client meeting is; it would be awesome to have something cool to show. The finished spike will be completed by Saturday.

There are three areas of the GUI: the left, center, and right. On the left is a list of all the different things that can go on a website (menus, images, panels, etc.). You can drag and drop these elements into the center area, which shows how the website will look. You can select elements in this working area. If an element is thus selected, the right area will show options for that element (styles and other attributes).

The idea is to do this for the specific example of menus.

Ian

Sunday, February 19, 2012

Getting Set Up

Hi,

After a major ordeal attempting to get stuff to work, I have finally succeeded.

Installing GWT was a hassle. The download link appears somewhat broken. In any case, it would get about halfway through, and then crash with a timeout error. I downloaded an archive, and installed from that without (too) much further ado (tweaking settings were necessary).

After this, I installed Smart GWT, which is a library over GWT (I think?).

After installing SVN (Tortoise) on my computer, I was able to check out the project's repository. There were two spikes. One ran and one did not. In any case, I'm fairly confident that I'll be able at least to use this thing.

Next time: my spikes and udpdates!

Ian

Thursday, February 16, 2012

Getting Started

Hi,

So I met with the group in the common area at Centennial. I like my group members, and I look forward to getting to know some of them more closely.

I received a sort of mental runabout trying to figure out the project's architecture, but I think I have a (better) handle on it now.

Both the group leader and myself agree that it would be good to design the project, and then divide work. With luck, this will be done soon.

We are tentatively planning to meet this weekend, and I think it would behoove us to do so each weekend. Due to some limitations, ideally this could be done remotely, at least sometimes.

The project will have a surprisingly large coding component. Of course, in Java. Le sigh. At least there's a reason: the Google Web Toolkit, which will be an important part of making the project, is based on working with Java. Given how awesome it is, I . . . suppose . . . it will be worth it.

For now, I'm having trouble installing some specific Eclipse plugins requisite for the project.

The group leader should have something akin to SVN up tonight (or very soon, certainly).

Ian

Monday, February 13, 2012

Project Chosen

Hi,

Well, I think nearly all of the really good projects got culled. None of the remaining ones interested me as much. In the end, none of my top five were chosen.

I asked, 3/4-seriously, if I could just go solo on my own project, but I vaguely caught a "no" from Ackley's direction.

All things considered, it's good, I suppose, that I got my first choice of the remaining projects.

As I understand it, it will be a mostly web-based project. This will provide an opportunity to brush up on my web design skills (although my new website has helped with this already). In particular, I look forward to exercising my PHP-fu, and gaining experience with JavaScript (I hope it won't prove as disgustingly condescending as Java).

My CSS skills are also pretty pathetic. I've used it, but poorly. In my new site, I have one CSS sheet. It's actually a PHP file that's loaded as CSS.

Anyway, though I'm not . . . ecstatic about how everything turned out, I'm going to make the best of it.

Ian

My Presentation Rant

Hi,

Well, now that it's over, my project pitch I think went pretty well.  I sat down (actually paced endlessly) and memorized the following tirade:
Hasn't it ever bothered you that those games you play are plotless, and that any given backstory is merely an excuse by the game developers to give you repetitive gameplay?

At the risk of sounding like an idealizing hippie, I find the dreary monotony of modern games despicable. Something needs to be shaken up, and I think that it's high time that that shaking up comprised the pathetic excuse of a plotline that as a gamer you endlessly have to suffer through.

Consequently, I am making a game where there is something to think about. It will be a WWI flight simulator, but the characters will have personalities, and the backstory will have merit in its own right. It won't just be some curtain to hide a gaping editorial hole as in some other game where you just punch the same buttons in the same ways to do the same thing for the same rewards, incentives, and dumb monologues.

By contrast—and, in case you glazed over it in my proposal—our game will have a plotline that is surreal, anachronistic, bizarre, and downright insane. We are going to make a game so “out-there” that when something like a swarm of giant spiders, quantum tunneling through mountains, wielding mayonnaise-shooting death ray cannons, accompanied by dragons, and lead by Dalek overlords under the mistaken geographic presumption that they're in Spain, kick down the front doors of indie gaming and crash the party, the rest of you will be too awestruck to wonder why you didn't think of doing it first.

Well, here's your chance. I need imaginative, flexible, fast-coding innovators with a passion for creating the next big thing out of whole cloth. It will be fun, challenging, and rewarding. I need CS students who thrive on their own reserves of such commonly untapped creativity. If you like code, and you like good games, we need you. Please consider my proposal; thank you very much.
The tirade was delivered confidently, and I'm pleased to say that that I exceeded my expectations for the amount I would remember (nearly all; faltered a bit at the end in the moment).

I got a lot of people asking me about the project and seeming interested.  However, I think the overall effect was too strong.  In one student's words: "Your project sounds really great, Ian, but honestly, I think you're too fucking insane to work with."

Well, I prefer the term "idealistic".  Oh well.

Ian

Wednesday, February 8, 2012

Revised Project Proposal

Hi,

Well, I basically rewrote the proposal.  It's still recognizable, but I made a lot of improvements that addressed most issues.

In particular, I've clarified the point about the story, and addressed redundancy by breaking the projecting into chunks and analyzing each of those chunks to completeness within that section.

The formatting is, I think, nicer, and I think the overall structure is much more coherent.

So, here's the link!
http://cs.unm.edu/~imallett/other/460/pp2f.pdf

Ian

Saturday, February 4, 2012

Overall Reaction to Reviews

Hi,

So, I've gotten all the reactions back from all people (two students, plus Ackley).

In general, I think that they provide some key insights, and give inklings of suggestions to improve the overall proposal.  I also feel like some people missed important and clearly obvious sections of the proposal, or were unnecessarily rude.

This aside, in sum, here are the (major) things that will be addressed in the final revision of the proposal:

  • Story: The story was truly intended to be open-ended, so that everyone in the team can share in making it awesome.  I VERY DELIBERATELY did NOT specify what the story would be, so that we'd all be able to work on it.  However, this point needs to be made much more clearly.  Every reviewer commented on this.
  • Missing Sections: The proposal has several missing sections, such as stakeholder identification and budget.  These must and will be added.
  • Consistency and Flow: The proposal is sometimes redundant and inconsistent.  Personally, I feel like the entire thing reads like a hippie's rant--disillusioned and illogical.  Other reviewers also noted that the proposal is often redundant.

Ian

Friday, February 3, 2012

Ideal Project Team in Three Words

Hi:

To me, an ideal project team is:
"Fast, Flexible, Professional"

"Fast": software development needs to happen quickly.  Changes need to happen rapidly so that progress can be made.  This goes hand-in-hand with flexibility to produce Agile Software Development.

"Flexible": The software team must be open to new ideas, and be able to quickly adapt to implementing them.  This ensures that the project can be restructured in the event of changes, or even completely scrapped and renewed if necessary.

"Professional": Finally, the software team doesn't merely need to be productive; the software team must be professional to both themselves and the client.  This ensures that there is a working environment for everyone that is conducive to actually making progress.

Words I might have included, but didn't:
"Competent, Intelligent, Cheerful"

"Competent": Good software development does not have to be made exclusively by competent people.  Although it is somewhat of a prerequisite, so long as the development is professional and fast, the product will be good.  This may imply that being able to develop quickly while still being professional implies competence; so we needn't be redundant by including it.

"Intelligent": Similarly to "Competent".  Additionally, while intelligence is important to being a computer scientist, I'd argue that being flexible and open to new ideas is more important.  Without an ability to learn or be open to new ideas, intelligence stagnates.

"Cheerful": One doesn't need to enjoy one's job to be successful at it (although I would hope that you do . . .).  Simply so long as the person's attitude does not cross the boundaries of professionalism.

Ian

Wednesday, February 1, 2012

Proposal Review 2

Hi,

The second proposal is entitled "SimpleMesh Project Proposal".

The gist of the second proposal is to create a very easy language (file format) that will allow novice users to write genuine 3D models easily.  Additionally, the project proposal supposes that this format can be easily read and written by scripts.  The second part concerns the creation of a model reader for this file type that allows basic navigation and other viewing.

My overall reaction is that the proposal has merit, especially in that is seems to be a (fairly) new idea (I myself have definitely found myself wanting something like this at one time or another).  However, I do feel like the proposal itself does not do the idea justice.  The paper is riddled with typographical and grammatical errors, and the proposal's structure could use some work.

Format: 4/10 = > 2.0/5.0
The paper is long enough that I feel like a table of contents might be useful.  There are a variety of headings used, inconsistently, to attempt to give some order to the paper.  However, the text itself seems haphazard.  I get lost in the middle trying to remember what exactly the purpose of a particular section is.  Some things would better be put in other places.  I support the author's inclusion of a terminology page to help readers unfamiliar with the topic area.

Writing: 4/10 => 2.0/5.0
The proposal has many errors.  I specifically noted many in all of grammar, verb agreement, punctuation, hyphenation, pronoun agreement, and sentence structure.  Spelling is mostly okay, although some typos manifest conspicuously throughout.  The writing itself is colloquial instead of professional, and reads like a transcript of a stream-of-consciousness monologue.  Although I previously mentioned this in the format critique, many sections feel out-of-place where they were put.  In fact, the entire thing doesn't appear to have been revised or proofread at all.  At least the general message is clear.

Goals and Tasks: 6/10 => 3.0/5.0
The author clearly has a good vision of what it is he wants to accomplish.  However, only some of this is successfully imparted to the reader.  The goals presented are at times vague, or alternately too precise (e.g., "the SimpleMesh project will consist of many adapter parts across many systems." versus "The orbiting distance[ in the viewer] can be adjusted by shift+up and shift+down.").  It should be said that everything the author wants done is present within the report; my complaint is that the goals can often only be inferred from the author's rambling prose.  My advice: short bullet points; expand as necessary.

Scope: 8/10 => 4.0/5.0
After reading the proposal, one actually has a surprisingly good idea of what the project entails.  I can't say I understand exactly what the boundaries are, because the project is deceptively open-ended, but I have a pretty good idea based on what the author intends to accomplish.  This is definitely something my report needs to work on too: I think an explicit this-is-in-but-this-is-out discussion would be beneficial.

Plausibility: 8/10 => 4.0/5.0
I think the project is definitely doable, but, like my project, I think a lot would depend heavily on the team.  I think that file parsing and semantics will be harder than the author anticipates.  Making the project general enough to be useful while still fulfilling its stated task of being incredibly easy would be a fine line the project would constantly have to tread (for example: "Can I use 3D textures?  Don't the texture coordinates have to be 3D?  Can I do that with your system?").  I feel like a more concrete plan would improve plausibility, although I may just be getting confused by the inconsistent language of the document.

Novelty: 9/10 => 4.5/5.0
The project isn't completely novel, but I think it would fill a much-needed void in graphics.  As I said above, it will be a fine line of complexity.  But, I think the idea is basically new to this field, and would be useful besides.

Stakeholder Identification: 9/10 => 4.5/5.0
The stakeholders are never identified explicitly, but the author does in fact state that the project is for people who know next to nothing about graphics ranging to people who just want quick and easy fixes for modeling applications.

Support and Impact: 8/10 => 4.0/5.0
The author identifies the project as open source and expects it to subsist, at least at first, on a PayPal donation button.  I've had a PayPal donation button on my webpage of free tutorials and projects for about five years, and--though those resources have found at least some audience--as of today, no has ever clicked it.  Not even once.  But, I think it might be plausible, and honestly I can't suggest a better option for an open source project, so I can't really complain there.  The section is brief and could be in more detail, but overall, I think this section is pretty good.

Evidence: 6/10 => 3.0/5.0
The author presents some information, but generally it expects readers to assume that such a product would be useful.  To some degree, this is obvious (both having experience in this field, and through reading between the lines of this proposal), but it's never clearly stated.

Challenges and Risks: 4/10 => 2.0/5.0
The author acknowledges some of the major challenges, including the difficulty of using 3D APIs, but this is only part of the project.  A huge part of the project seems to be the .smpsh file format they plan on inventing from scratch.  This is not even mentioned as a challenge.  Lexical analysis and parsing of custom file formats, while certainly possible in this time period, is still a significant challenge that should not be taken lightly.  The author also mentions the difficulties of making a website and making sample meshes.  I agree that these are challenges, but like all the challenges of the project, the discussion about them--and more importantly how to solve them--is either lacking or nonexistent.  The author stresses creativity as a requisite for overcoming challenges.  That's good, but it doesn't really address the specifics of what will make this project challenging, nor how to address those concerns.

So: overall, I think this is a comparatively excellent idea.  As stated above, I think that the proposal could benefit from being restructured and cleaned up in the ways enumerated.

Ian

Project Proposal Review 1: Response

My project was reviewed by Kyle Wagner on his blog, at this location.  I think he raises some good points, but has also missed some.  I'd like to comment on his response below:

Kyle writes: "The proposal should have focused on making a game engine rather than have the pretense that a story driven game is made.".  This is my bad.  Apparently, the proposal did not give the impression that the point was on making an involving story.  The point really is to make a story-driven game, but I suppose I must have focused too heavily on implementation details.  This will be revised in the next version.

Kyle's opinion is that the project is "impossible to implement in a twelve-week period".  To this allegation, here is a screenshot from a one-day "spike" demonstrating the feasibility of the project.  Start to finish, the spike took 22 hours:
Notice the crude shadows.  The airplane is also shooting, and flying over a terrain loaded from a file.  The code for this is horrendously messy, but it demonstrates that the technical challenge of making a basic game framework to add content to is NOT impossible.  I also found the assertion surprising given our mutual experience in CS 413, which required a demanding project as well.

It should be noted that in-class, Kyle said that making a renderer might not be such a difficult task for someone like him or me, but that as a group project proposal, it might be unfeasible--although I do say in my proposal that "A graphics library will likely be used to help set up the basics.", and I would hope that people would know their capabilities before signing up for any project.

Follows, a breakdown of Kyle's quantitative assessments and my responses to his rationale.

Format: 2/5
Kyle says that the format is hard to follow and disorganized, as well as missing sections.  This is somewhat true.  I feel like the proposal is well-organized, but some sections are indeed missing.  This is because I was unaware of Ackley's recommendations for the proposal here.  In any case, this is an excellent (implicit) suggestion; this shall be corrected in the next revision.

Writing: 3/5
Kyle writes that the grammar and spelling are "fine save for a few errors".  I'll look for these while revising.  Kyle points out that each section is a rehash of the last.  This is somewhat true; I tried to look at the project from different perspectives in each section.  However, it is correct that that could be improved.

Goals and Tasks: 1/5
In contrast to Kyle's assertion that there is no description whatsoever of how the project should be accomplished, the proposal does give more than two pages of step-by-step discussion about what each phase of the project entails in section V.  I agree that the "proposal gives a vague idea of what the author wants"; my idea is never really concretely hammered down.  This will be revised.

Scope: 1/5
I agree that I can be more clear about what is and what is not a part of the project.  I feel like there is at least some consideration (for example, networking is optional if there's enough time).

Plausibility: 1/5
Kyle's opinion is that "there is a lack of direction" and that "the project has too many ideas crammed into one to be plausible".  I definitely agree that there is at least some lack of direction, though I note that I deliberately left the storyline open-ended, because I think that's the fun part of the project, and I didn't want to tie us down.  To the second, I don't think there are too many ideas--the project is, in essence just a WWI flight simulator with a story that doesn't suck.  The only extra idea--to make it multiplayer--was deliberately made optional in case it would make the project too hard.  And, with regards to plausibility, please see the screenshot above.

Novelty: 1/5
Kyle writes that the idea is too unclear to be compared, and that there are no competitors listed.  I think that this is true.  In the next revision, competitors and inspirations will be listed.  I would argue that my project has at least some novelty, in that it goes in a direction that modern games never go, but I think perhaps this was impeded by a lack of clarity.

Stakeholders Identification: 1/5
I did not list stakeholders in my proposal.  This will be corrected in the next revision.

Support and Impact: 2/5
Kyle writes: "The author throws out a couple of ideas[,] but never flushes them out to be a business model."  I completely agree with this.  It shall be corrected.

Evidence: 1/5
I also agree that there is a lack of evidence in the arguments for need and impact.

Challenges and Risks: [No score given?]
Kyle writes "The proposal doesn't take a look at what might be difficult or why it is."  I would like to direct Kyle's attention to section IV, "Requirement Challenges".  This section describes the major issues and potential resolutions of each of the six major portions of the project, over the course of a solid two pages.  I actually think this is one of the areas the proposal is best on.

So: thanks, Kyle, for reviewing my proposal!  I appreciate that you expounded the faults of the project proposal as you saw them rather than trying to sugarcoat or ignore them.  And, despite the fact that your review was harsh (even you said as much), I think that it's all for the best.  I feel like a lot of your criticism is justified and useful, although I think that you missed some key points when reading.  I will take your recommendations into consideration when revising this proposal.

As for me, I realized some other faults in my proposal as I reread it with Kyle's critique.  These issues will be fixed as well.  A selection is below:
-As opposed to listing what is included in the project and then letting the reader infer from there, I think a good idea would be to define some negative boundaries of what will NOT be in the project.
-Adding explicit risks would be beneficial in section IV.
-Kyle mentioned this in passing: there is no budget analysis.

Ian

Monday, January 30, 2012

Proposal Review 1

Hi,

I've agreed to review two proposals.  The first is entitled "Healthy People".

The gist of the proposal appears to be to create an app for the iPhone, iPod, and iPad that gives lifestyle suggestions given user data, with the intention of helping people lose weight.

My overall reaction is that the proposal might be a good idea, but that the quality of the presentation . . . needs work.  The proposal lacks structure and is at times redundant or incoherent.  The author only really addresses one of the issues and does not present a clear plan for project execution.  Overall, the project seems like a simple enough idea, but the proposal remains unconvincing.

Format: 3.0/10.0 => 1.5/5.0
There are no marked sections; the entire proposal appears to be a single stream-of-consciousness rant.  Upon closer examination, the proposal does possess a loosely delineated structure, with discussion of various aspects of the project being in generally related areas.

Writing: 2.0/10.0 => 1.0/5.0
I don't really think judging on this is fair.  From the way the paper was written, it is clear that the author is an ESL speaker.  Consequently, I don't hold faults in this area against the proposal.

Goals and Tasks: 4.0/10.0 => 2.0/5.0
The project is roughly defined.  By the end of the proposal, it is clear what the project would entail, even if it is not completely stated.  There is only a very limited breakdown of what would be required, but the goal is present.

Scope: 6.0/10.0 => 3.0/5.0
The project has a decently defined boundary, in the sense that the entire project idea was stated, and boundaries can be inferred from that.  However, the description is somewhat vague, and only some aspects of implementation can be inferred.  There's not really a concrete sense of what to do, with scope only arising from a description of what the application's behavior would be.

Plausibility: 8.0/10.0 => 4.0/5.0
The project appears to be simple enough to implement within 12 weeks.  However, the author admits that he has no knowledge of development on any Apple platforms, nor equipment to test it on.  Programming for Apple mobile devices typically requires Objective-C, which is a huge pain to work with, especially if you never have before.  It is only because the project is reasonably simple that I think it is still doable.

Novelty: 7.0/10.0 => 3.5/5.0
Exercise programs aren't really new.  The author acknowledges this.  The differentiating factor according to the author is that the proposed project would not give confusing data that would confound the user (heart rate, calories used, etc.).  Instead, the author suggests making the UI simple, with only lifestyle suggestions and weight loss being reported.  This isn't really novel, but it has merit in that many people don't really care about numbers and statistics, and just want some suggestions, and it is not an approach that is typically used.

Stakeholder Identification: 9.0/10.0 => 4.5/5.0
The product is clearly intended for those overweight or concerned about their health.  It does not explicitly list these as the target audience, but the intention is nevertheless quite clear from context and remarks within the proposal.

Support and Impact: 4.0/10.0 => 2.0/5.0
There is no mention of anything about supporting the project.  As close as it comes is "it will not have cost"--so, open source?  Free?   No support mechanism is proposed.  However, the author consistently alludes to the notion of helping people and making a difference in their lives, so there is some implicit notion of making an impact.

Evidence: 9.0/10.0 => 4.5/5.0
Actually, the proposal opens with a statistic concerning the amount of overweight people in industrialized nations.  The impact is fairly obvious by the project description.

Challenges and Risks: 4.0/10.0 => 2.0/5.0
The author acknowledges the main issue: that he knows nothing about application development on Apple platforms and currently has no resources to do so.  Unfortunately, there is no apparent realization that any research (which would have shown that this may be harder than first imagined) was conducted.

So, overall, weight loss programs aren't really my thing, but I can see the idea having some use to some people.  However, as presented, the proposal does not effectively sell itself.

Ian