Does anybody remember unit tests? But also a wicked kickin' readme!

The week that ended July (and from the weather patterns, ended summer as we know it also) was a little off-balance in terms of rhythm, with project lead David Humphrey away on vacation, his presence is clearly missed and noticed when it comes to uniform progress as a whole with the Mozilla Webmaker team.

Nevertheless, my fellow researchers and I met the issues to be faced this week with fierce tenacity and ambition to further lessen the remaining bugs in MakeDrive at its current state. The first half of the week preoccupied me with more unit test patches to land, with most of my time spend on a patch dealing with having to redesign some of the callback function signatures in the tests' infrastructure to cater to Node.js callback parameter conventions. Debugging galore ensued in order to correctly trace and follow the data passing inside of the callback hierarchy, but it ended up being invaluable learning experience.

Finishing the week, I took on the task of implementing the first comprehensive readme document for upcoming first users of MakeDrive. While initially daunting, this was accomplished with the help and insight of every member of the team pitching in on their section with their expertise and I believe that the final result speaks for itself.

This week will primarily concentrate on catching up with stress-testing Nimble with MakeDrive on the deployed page fellow team member Ali Al Dallal has up on the web. This will also be a wonderful opportunity to familiarize myself with emerging JavaScript and HTML5 technologies that Mozilla is already beginning to implement in its products and services such as Angular.js. 

Query Strings and more Unit Tests

While front-end work is always fun and demonstration-friendly more frequently than functional coding, all those pretty icons, animations, and colour schemes wouldn't be very useful without any backend code to provide purpose for their presence.

My work was focused on implementing pre-production logic in the session/authentication data handler functions to accept query string data (that comes in on the address bar) as well as standard cookie data. This was added to increase the flexibility of MakeDrive to be able to cater to such things as firefox extensions sending incoming verification data.

The rest of last week revolved around adding more unit tests to increase the comprehensiveness of the existing test suite in conformity to the complete overhaul of the client and server communications that are now completely reliant on Websockets. I was particularly concentrated on adding more test cases for the sync messages being passed back and forth.

In the usual cadence of things, overall the week was very tasking but ultimately very productive. MakeDrive is just about ready to be deployed out to the public, and after a design overhaul by the Mozilla UI/UX team, Nimble will follow soon thereafter.

Bug Squashing, Issue Triaging, and Nimble UI Enhancements

Communal elation in the group is still very apparent after the functional demo of Nimble and Makedrive working together, and we are all focusing that positive energy to keep a rigorous pace in order to arrive at the upcoming milestone this Friday revolving around getting MakeDrive to be stable enough to deploy to the public and be used in non-controlled environments such as the other Webmaker tools.

Last week, I created some fun and practical extensions to the front-end UI in order to test the Brackets appshell's potential in its current form in the context of being able to manipulate or change the end-user interface without having to change any of the code already implemented. I went ahead and recorded a video demo of my results on YouTube:

I can't help but feel proud of what little front-end programming prowess I've managed to cobble up, haha.

My focus this week veers back to backend functionality with more bug squashing on the MakeDrive end of things, particularly in the scope of webmaker authentication. I will be tackling some code removal/refactoring to eliminate unnecessary or arbitrary module imports and process executions as well as attempt to plug in support for query string session data as an alternative to cookies in order to extend webmaker-auth's login methods to be able to use firefox extensions and the like. Much learning will likely be had.

As always, stay tuned for more updates!

Realizing the Vision and Beyond

I couldn't find a better way to present this besides show and recommend anyone reading this to check out this YouTube Link. Everything this team has been working on for the past 2+ months now functionally amalgamated and in a state where the world can start seeing it. Nimble (Brackets) in the browser using the MakeDrive filesystem to sync files between active sessions of the same client. Extremely exciting!

The rest of the summer is about polishing and perfecting the operation of the project and adding features in order to really turn it into a bonafide Mozilla product that becomes a welcome addition to the rest of the Webmaker toolkit.

Nimble and MakeDrive's Future

As expected, with the help of the Webmaker team, I managed to finish a functional proof of concept implementation of the Websocket authentication module that Alan Glickman, Kieran Sedgwick and I planned out the week prior just in time to quickly demonstrate it on Tuesday.

On Thursday, our team also presented the current state of MakeDrive thus far and what has been accomplished up until now with the project. While the lack of practice in the areas of structuring roles and memorizing who begins what part of which area of focus left a bit to be desired, it was received with healthy amount of praise nonetheless.

After planning for the rest of the week and next week's tasks, Gideon Thomas and I began to pair program converting the client-to-server communications of MakeDrive from SSE's to Websockets. A very productive week on the whole for everyone on the team, and we're all hoping to end next week with a functioning instance of MakeDrive running inside Brackets on the browser for project lead David Humphrey's last week of working at CDOT for the summer. Fingers crossed.  

The Wonders of Mozilla Proper

Last week, the team managed to finish the functionality for bi-directional syncing with MakeDrive on the browser, that being a huge milestone in the Nimble project. Helping in the design of the front UI, work on unit tests, and pair program through bugs was as rewarding and productive as it could've ideally been.

The cherry on top of the icing was being able to visit the Mozilla office downtown to present the demo on Friday. The working space in and of itself is worthy of song and film, with ping pong, a music corner, couches abound, snack and fruit bar, and an espresso machine that cannot get enough praise for its impeccable quality. More importantly, to be able to gain insight from and work with some of the most talents minds in the industry was invaluable to say the least.

With project lead David Humphrey expected to return from vacation this week, there's lots of catching up to do with Websockets authentication and changing the client codebase to function completely off of Websockets instead of server-sent events. Another daunting week ahead, with newfound energy and inspiration to tackle the tasks ahead. 

To Websockets or not to Websockets

With project lead David Humphrey currently on vacation and senior team member Ali Al Dallal called away to Vancouver with tertiary work from Mozilla, there was no shortage of work to be done or tasks to be undertaken. Kieran Sedgwick went ahead and took over Makedrive unit testing for the time being in order to try and eventually successfully solve the nightmarish bugs that blocked the tests' infrastructure from being fully implemented and able to support comprehensive codebase testing.

Concurrently, I went ahead and switched gears and focused on the two tasks of trying to research and piece together a proof-of-concept of a Websocket-core API system that handles user and session authentication before automatically upgrading the connection protocol from HTTP to WS. This is still a work in progress, but much learning is being had thus far. Some decisions might need to be made on the issue of the potential limitations of the core library not being able to handle upgrade events as comprehensively as might be necessary in order to properly and securely create this connection switch validation.

Lastly, I was helping fellow team member Gideon Thomas with the planned upcoming demo of the bi-directional syncing functionality of Makedrive, mainly with designing the front-end UI and pair programming through the client-to-server communication and invocation of our libraries. That unfortunately wasn't able to be materialized due to the discovery of a bug with the server-side diff route validation that is still being solved to this day which not only affected bi-directional syncing but the unidirectional syncing that was demoed weeks earlier as well. Slightly heartbreaking, but on we fight with the knowledge that eventual victory will taste that much sweeter.

Makedrive Unit Testing

In most projects of similar breadth, understanding the big picture is paramount in the search of perfecting the design and function of the system itself. This week was crucial in my comprehension of how all the pieces need to integrate into the final product in order to satisfy all the desired end-user applications of the APIs that we're creating.

My assigned work was primarily focused on unit testing the sync route hierarchy that takes place when the end user initiates a file save from one of his active sessions. This is the perfect task for mastering one's familiarity with a complex codebases. A firm grasp of what each variable, function, and object contains and passes is paramount to successfully and efficiently test against the environment. On top of polishing up agnostic unit testing syntax and conventions, I inevitably have to traverse through all of the code in a way that isn't necessary to accomplish most other work. This is turning out to be an extremely fascinating and challenging endeavour, since valid user credentials from the client session as well as valid session objects including file and directory data must be present and persistent in order to initiate the sync sequences and were therefore arbitrarily mocked in order to successfully go through each route. This infrastructure was developed by project lead David Humphrey due to the time constrained nature of the current sprint, but the helper functions that I wrote and am presently using ended up emulating the same behaviour in order to achieve the similar goal of having to persist valid session data throughout the test process.

Backtracking to more fundamental concepts that I must still catch up on, I've taken the weekend to fill in some knowledge gaps with object context and scope in javascript. A great article I've found on this gives insight on this from the simplest to the more realistic use cases can be found here.

With the first of the 4 routes seeming to be properly implemented, and with the help of fellow teammate Kieran Sedgwick, I intend on finishing the test foundation for all of the primary sync routes by the end of today and start integrating this logic with rsync and websocket applications for the rest of the week.

The Acclivity that Separates Programming from Development

Concluding this week was the presentation of Webmaker team's first completed two-week-long sprint which showcased the initial integration of makedrive with filer and its initial connection with client-side DOM sessions. As I alluded to in last week's post, my work on this heartbeat was primarily focused on implementing Server-Sent Events.

The design was structured as so:

(This was also seen in my original proof of concepts' front-end page). The initiating client session would "save" a file they just created, effectively sending a request to push the data into the server. Once validation and syncing would be completed, I (along with great help from Mr. Sedgwick, yet again) used a server-sent event back to all the other active client sessions (on the same user) that the sync and push has successfully completed and that it's time for them to update their version from the server. Completing the cycle and syncing all the other sessions is one of the goals of the upcoming sprint we are about to undertake.

The demo, although executed wonderfully as a concerted team effort, ended up being received with a rather tepid response. This was likely due to the majority of the mozilla developers being away at conventions, and leaving the large part of the crowd to be more community and marketing work-oriented folks.

This week I will be working on writing server-side and restful-API unit and functional tests. More to come next week, as usual.

Server-Sent Events and Bootstrap Tinkering

Last week was definitely substantial in terms of activity. In the pursuit of surprise, however, summary and descriptions of my work on the current sprint will be completed on this upcoming week's blog post when initial implementation into the MakeDrive logic will occur (hopefully by week's end) and be visually demonstrated first.

I am delighted to report about my initial fiddling work with Bootstrap CSS which is coming along rather swimmingly and excitingly. This front-end resource is being used to present my first proof-of-concept of the relatively new SSE interface. Documentation is readily google-able and as in-depth as you need it. When used in conjunction with JQuery, it provides for extremely efficient element generation and manipulation. You can event grab pre-made templates from the main website and customize them from there for web design on the fly. It's front-end beauty that's ideal for the timeframe of back-end coders.

Friday also welcomed a new workshop to host at Silver Springs public school. Very similar in structure to the workshop held one week prior, with the addition of me personally spearheading the last one of the day. The students were nearly as attentive and focused as the first school, and showcasing Webmaker proved to be a success yet again, with excited praise coming from faculty and participants alike. One trend fellow teammate Kieran Sedgwick and I were noticing is the slight difficulty for the younger audience to immediately find the tools they were looking for from the main page. Kieran has already filed a new issue on Github for the front-end team to take a look at.

Lots more news to come after the shipping of the first sprint by the end of this week!

The Plugin, the Sprint, the Speech, and the Workshop

To use a word such as 'eventful' for this week would likely be the understatement of the century. In the short span of 5 days, the Webmaker team at CDOT managed to implement and demo a barebones functional version of a Brackets extension called Wavelength, and crack down on the remaining issues in the Filer codebase in order to ready it for porting into MakeDrive. In between all of this, fellow team member Kieran Sedgwick and I were sent out on our first workshop for the Toronto District School Board, and earlier in the week I was able to harness my communication skills and impromptu charm by welcoming a visit from high profile individuals.

In the spirit of honesty and accuracy, the work on the brackets extension did start on Friday, but that shouldn't take anything away from the tenacity and talent of the team being able to dive straight into a brand new API and push up a functional extension within less than 10 hours of work per person. Credit and thanks should be given to Adobe for not only creating sufficiently thorough documentation to peruse through in times of need, but for uploading templates and specific examples for starting to build extensions that proved to be paramount in our ability to create ours in such a demanding timeframe. My particular contribution was implementing the toolbar icon and the events necessary to emulate the standard behaviour relating to mouse movement and action - changing background colour when hovering over the icon and changing the icon's colour when clicked on or activated. The biggest logical hump for me to overcome was having to wrap my head around the fact that Brackets elements are all effectively DOM elements; I was looking for an API-specific function or parameter that would invoke or manipulate the toolbars when in actuality these are all DOM elements controlled by standard javascript calls and CSS classes. Extremely neat.

Filer work made up the bulk of the development work for the week, and the experience was the first true test of what is likely to come in the near future in regards to the independent workflow process. When the rest of the team was focused on their own respective tasks, and project lead David Humphrey is juggling 50 other issues at any given time, I was largely left up to my own devices and ken in order to solve any blockers that obstructed my progress. IRC is always there, but honestly it could never amount up to the quality of a peer's physical presence. Quite overwhelming at first. Filer's codebase is relatively vast in comparison to what I am used to working with up until now, and file system logic is brand new territory for me. Combine those factors with a rather outdated documentation and the questions started piling up quickly from under the woodwork. Once I gathered enough context about the variables and functions involved and more insight into assertion-agnostic unit testing, everything else eventually fell into place with the exception of a few kinks. Testing the logic proved to be a challenge as well since I was requested to run the unit tests on a local server instance to emulate an environment that accommodates CORS mechanisms. Mac OS X builds have apache2 built in to serve webpages locally, but being able to properly implement that also ended up needing the seasoned and extremely capable hands of Chris Tyler, OSTEP team's project lead and veteran linux wizard. Apache2 is overly restrictive in its document path hierarchy and file permission structure. I initially thought that placing a symbolic link of my index.html entry point in the default path given in the httpd.conf file. That proved to be unsuccessful, so Chris needed to change the default path to start at my Document file tree and set the chown group of all the inner files and folders to the "staff" moniker in my case with lastly the addition of granting read and execute permissions to the "other" octet (chmod 755, or similar). Allowing symlinks is apparently a dangerous course of action that opens your files to attacks. All in all, I managed to send pull requests of two issues related to filer by the end of the week, ultimately surpassing my own goals in the end.

Thursday was reserved for the workshop activities at Kennedy Public School for a grades 6-8 career day. Kieran and I agreed on engaging the children in a relatively simple task of creating a webpage with their own background colour and URL-sourced image anchor using Webmaker's Thimble editor. Initially, we believed there would be enough time for the students to look for and find the syntax required to achieve the task on their own, but for many of them the learning curve was a bit much and we quickly adapted ourselves to nudge and help them along in the right direction. Nearly everyone was attentive and listening, and we were pleased to see some cases of genuine interest and comprehension of what they were looking at and doing. It was a smooth, productive day and I'd like to think that we've helped nurture the future software development giants of tomorrow some way, shape, or form.

I conclude with a pleasant surprise this week when one of the ICT professors dropped by the office with a pair of executive academic representatives. I had the opportunity to give them a quick overview of our project, and tried my best to add as much genuine personality into the conversation as I could while keeping a professional manner. It's wonderful experience for anyone who wants to refine or nurture their interpersonal skills in a more improvisational dynamic. These kinds of meetings often lead to invaluable networking channels that will reward you in waves later on. Wonderful stuff.  

A Trial by Fire - Initial Exposure to Bower, NPM, and require.js Modules

Starting the first week in any place of work tends to be chaotic to say the least - settling into the expected workflow and output, setting up your environment, and getting to know your team members, if any. The first week's project was designed to expose and introduce the popular and emerging web technologies used to test, build, and publish web apps in the front and backend of systems.

Node.js functions and objects were used to import and export a simple module that parses through incoming data in markdown file format and return the URL addresses of the links found in the strings. Initially, the data was placed in a static variable in order to focus on the pure logic of the function that parses through the data. Unit testing was created with mochaTest and automated with a Grunt task, alongside JShinting and linting for any syntactic concerns. Travis was used in conjunction with the Github repository currently being worked on in order to invoke all of the testing tools anytime a change is pushed, and the results would be emailed to the developer shortly thereafter. After those dependencies were properly implemented and tied together, the module was published to the npm and bower registries in order to be useable by the public. Bonus tasks were to implement logic to read data from incoming file arguments on the command line as well as parse data from a front-end web app using require.js to import the module and its dependencies.

Managing to accomplish all the aforementioned goals save for the front-end medium concluded a work-intensive and very intriguing week. Almost all of the technologies used in this project were brand new and my exposure to them will hopefully prove to be invaluable when tackling the coming work ahead with the Mozilla Webmaker team to release a functional version of the nimble to the Webmaker toolkit by the end of the summer. Any of the expected blockers that I've come across were efficiently quelled by the readily available amount of help from the rest of the team and the documentation of the APIs. I enter this week ready to tackle whatever challenges that cross my path next with cautious optimism and palpable excitement.