Git Template Application Repository

I titled this post what I wanted, but the result wasn’t quite so succinct.

What I wanted was a git repository that contained a template for future applications. I have done so much what I’m considering good work with Quizmo, I wanted to create a skeleton framework that I could fork into applications. So all of the components that I’ve created could be contained in one place and when I update them in Quizmo, they would likewise be updated in the template repository, and future applications would be able to easily pull down those same changes from the template repo. So I could really live the dream of rapid development RoR has been trying to sell for the last decade.

My first thought was to create a repo that had a dummy application in it and just forking it. Then I should be able to pass things back via the upstream and down with merges. It seems like such a simple operation. Unfortunately, that’s not how forking works. The problem is if you want to merge upstream, you have to merge everything. You can’t just merge the things that are in the upstream, there’s no mechanism for adding or committing one revision without committing every revision prior to that one.

So the only way to have different components connected via other repositories is to have multiple submodules. Everything needs to be contained in its own directory. There’s also fake submodules*. Which just relies on a lot more setup, I haven’t encountered problems with using submodules yet, so that isn’t an issue.

Instead of having the awesome application template repo I wanted, I’ve got a bit of a mish mash which currently looks like this:

The most important thing with this discovery of using a mish mash of submodules is that I should modularize my code. That is to say I need to have all related components in the same place. That seems obvious, but when going with the flow of yii, I put things where yii wanted them, not necessarily where they would be best served. Like twitter bootstrap, which got just strewn about all over the app.

The authentication abstraction I’m using is an IdentityFactory that chooses which xIdentity component to use which is an extension of UserIdentity but each individual xIdentity calls on its respective extension. So do the xIdentities go with the appropriate extension or with the IdentityFactory/UserIdentity? Asking the question makes me think it has to be with the extension.

Interesting links:

* Update: Upon working more with submodules, and paying attention to the pitfalls (outlined here) I’ve actually become very opposed to fake submodules (see above). Fake submodules relies on the person checking out your code to check everything out, and leaves the repository with no explicit link to the submodules. You basically just have to know. Which is fine for one guy working on one project, but is horribly irresponsible for a developer working for anyone other than themselves.

Peer Review, Peer Grading

With all the talk of MOOCs (edX / Coursera), I’ve been very interested in finding more information on peer review. So I’ve been reading the studies that espouse the benefits of peer review in general.

Duke Chronicle: Peer grading experiment a success, professor says
Mostly older articles via google

And the pitfalls:
How accurate is peer grading?

A couple years ago I was put in charge of working with UCLA’s Calibrated Peer Review for Eric Mazur. He was really excited about it — I was less so. But my problem was I was looking at the application, not the concept. Just because an application is overdeveloped drivel doesn’t mean what they were trying to do isn’t awesome. I’m of the thinking they should have simplified it. That seems to be the case with just about everything I see. Applications shouldn’t be as complicated as they’re made. The problem is there are usually too many people involved in a project’s inception and everyone needs to put a piece of themselves into it. But I digress. edX will be great.

I don’t think Mazur used the CPR for more than 2 semesters. Probably because there was too much overhead and it wasn’t intuitive enough. But a poor implementation doesn’t mean a poor idea.

Or at least that’s my theory on this. I hadn’t seen any progress with online implementations of this, people haven’t been pushing this teaching technique yet and it’s disappointing (or telling).

Coursera is making a run at it now. That is encouraging. That means edX will probably follow suit with a similar implementation. And I’m planning a mild implementation with Quizmo.

Using Smarty variables inside Javascript

I looked for appropriate answers for this on the smarty site, and the resounding answer was escaping javascript’s {}s with {literal}’s

Smarty template vars and Javascript functions
How to use Javascript codes in template files?
Access Smarty value with javascript

They basically all say to do something like this:

// $question_ids_json = ["1","2","3","4","5"]
{literal}
<script>
$(document).ready(function(){
	var question_ids = {/literal}{$question_ids_json}{literal};
	
	alert(question_ids[0]);

});
</script>
{/literal}

So they want to put everything in a literal tag, and then basically escape that literal tag when smarty needs to be used.

I didn’t like this, so I put the value in a hidden input and then just retrieved that input in the javascript, thereby keeping the js clean of smarty shenanigans:

<input type="hidden" id="question_ids" value='{$question_ids_json}'/>

<script>
$(document).ready(function(){
	//var question_ids = {$question_ids_json};
	var question_ids = eval($('#question_ids').val());
	
	alert(question_ids[0]);


});
</script>

But this is just a matter of personal preference.

Abstracting schemas for MySQL and Oracle with Yii Migrations

Yii migrations are an interesting way to keep track of your database changes.  I was originally looking for a tool that would abstract the schema SQL — with something like the Yii ActiveRecord — so I wouldn’t have to write duplicate code for Oracle and MySQL.  I mean, that’s kind of the point of going with the PDO abstraction.

So more than a few forum question/answers led me to Yii migration:

http://www.yiiframework.com/doc/guide/1.1/en/database.migration

I seriously have that page open 3 times in my tabs and I have read it top to bottom probably 20 times.  I keep going back to it over and over hoping it will have more information.  It’s just a little short on answers to all of my questions.

As I said, all I wanted was a way to abstract the schema, so the original premise of “migrations” was a little different than what I was looking for, but it being the only non-custom option, I went forward with it.  The point of migrations is to make sure the state of your database matches the revision of your code.  So it’s sort of like its own hokey versioning system.  As such, it doesn’t function as simply as I might like it.  It’s not easy to focus on one migration if it’s not the last migration.  Because the point of migrations is that you’re moving forward and editing a past migration would be like editing revision X without it effecting all of the future revisions.  So it makes sense for what it is, it’s just awkward.  For one the file structure HAS to include the timestamp.  And it has to include that timestamp at the front of the filename, so autocompleting these migration filenames is a pain in the ass.

One important thing they don’t talk about in the documentation is that the list of what’s been migrated is added automatically to your database via the “tbl_migrations” — and please remember the “s as it will insert it as lowercase, which could be annoying for Oracle users, though Yii seems to handle it “appropriately”.

The only major hurdle here is dealing with autoincrement. The way this was dealt with was I created an Autoincrement class that I plopped in the migrations directory:
protected/migrations/Autoincrement.php

This meant that I could just create the tables like so:

class m120402_194059_Quizes extends CDbMigration
{
	public function up()
	{

		$this->createTable('QUIZES', array(
			'ID' => 'pk',
			'COLLECTION_ID' => 'integer NOT NULL',
			'TITLE' => 'string NOT NULL',
			'DESCRIPTION' => 'string',
			'VISIBILITY' => "integer NOT NULL",
			'STATE' => 'string',
			'SHOW_FEEDBACK' => 'integer',
			'START_DATE' => 'datetime',
			'END_DATE' => 'datetime',
			'DATE_MODIFIED' => 'datetime',
			'DELETED' => "integer NOT NULL",
		));	

		Autoincrement::up('QUIZES', Yii::app()->db->driverName);


	}

	public function down()
	{

		Autoincrement::down('QUIZES', Yii::app()->db->driverName);

		$this->dropTable('QUIZES');

	}

Yii fixtures with Foreign Keys

CDbException: CDbCommand failed to execute the SQL statement: SQLSTATE[HY000]: General error: 2292 OCIStmtExecute: ORA-02292: integrity constraint (QUIZMO_DEV.FK_ANSWERS_QUESTIONS) violated – child record found
(/web/quizmo/PDO_OCI-1.0/oci_statement.c:142). The SQL statement executed was: DELETE FROM “QUESTIONS”

Totally annoying. Now I’m not one who has a ton of experience with foreign keys as most places seem to have relations with tables but never explicitly link them as such.

So a FK just doesn’t let you add a record when the record it’s linking to doesn’t exist and won’t let you delete a record when there is some other record linking to it.

The problem is I don’t think Yii thought about these when they did fixtures. They built in a way to deal with it, but it’s a little annoying.

Let’s ignore the fact that all of my tables are in caps because I want this to work in Oracle without having to throw single quotes around everything all the time. Let’s take a look at the fixtures of a section of my tables in a project I’m working on right now.

fixtures/
USERS.php
COLLECTIONS.php
USERS_COLLECTIONS.php
QUIZZES.php
QUESTIONS.php
ANSWERS.php

First I need an init.php in the fixture directory. This is called from /yii/framework/test/CDbFixtureManager.php


	public function prepare()
	{
		$initFile=$this->basePath . DIRECTORY_SEPARATOR . $this->initScript;

		$this->checkIntegrity(false);

		if(is_file($initFile))
			require($initFile);
		else
		{
			foreach($this->getFixtures() as $tableName=>$fixturePath)
			{
				$this->resetTable($tableName);
				$this->loadFixture($tableName);
			}
		}
		$this->checkIntegrity(true);
	}

So the init file just needs to do this:

foreach($this->getFixtures() as $tableName=>$fixturePath)
			{
				$this->resetTable($tableName);
				$this->loadFixture($tableName);
			}

The issue is the resets and loads need to be done in the right order. So the way I went about this is I put in an array of the tables, using that and a reversed version of the array.

$reset_order = array(
	'USERS_COLLECTIONS',
	'ANSWERS',
	'QUESTIONS',
	'QUIZES',
	'USERS',
	'COLLECTIONS',
);

$load_order = array_reverse($reset_order);

foreach($this->getFixtures() as $tableName=>$fixturePath){
	if(!in_array($tableName, $reset_order)){
		throw new CException("Table '$tableName' is not in the reset_order.");
	}
	if(!in_array($tableName, $load_order)){
		throw new CException("Table '$tableName' is not in the load_order.");
	}
}

foreach($reset_order as $tableName){
	//echo("resetting $tableName\n");
	// this runs the TABLE.init.php if it exists
	// otherwise it just does a $this->truncateTable($tableName);
	$this->resetTable($tableName);
}
foreach($load_order as $tableName){
	//echo("loading $tableName\n");
	$this->loadFixture($tableName);
}

That’s not all though, because EVERY table needs a TABLE.init.php file that will be run when “resetTable()” is run in the above script.

fixtures/
init.php
USERS.php
USERS.init.php
COLLECTIONS.php
COLLECTIONS.init.php
USERS_COLLECTIONS.php
USERS_COLLECTIONS.init.php
QUIZZES.php
QUIZZES.init.php
QUESTIONS.php
QUESTIONS.init.php
ANSWERS.php
ANSWERS.init.php

Without these scripts, each table will just DELETE FROM MYTABLE when resetTable is called. The problem with this is if you DELETE FROM COLLECTIONS without deleting the QUIZZES first, you get an integrity constraint violation. So a table init file needs to truncate all tables that are “children” of that table in the order of smallest to largest. COLLECTIONS.init.php looks like this:

truncateTable('USERS_COLLECTIONS');
$this->truncateTable('ANSWERS');
$this->truncateTable('QUESTIONS');
$this->truncateTable('QUIZES');
$this->truncateTable('COLLECTIONS');
?>

Yii handling “getLastInsertId” with Oracle

With MySQL or SQLite, when you insert something with an auto_increment field, it will automatically deal with the ActiveRecord by putting the last inserted id into $model->id. Because Oracle needs sequences and triggers to deal with that, neither the PDO driver nor the Yii PDO code felt it necessary to deal with that. Most people are probably fine with commiting to a database and throwing in Oracle specific sequence_name.nextval.

I just added a class in the models to extend CActiveRecord. The only thing you need to do with this is have your model extend this instead of CActiveRecord and add the variable schemaName to the model:

class User extends QActiveRecord
{
	public $sequenceName = 'USERS_SEQ';	
...

Yii Migrations: Dealing with auto_increment for Oracle and MySQL

Oracle deals with auto_increment by using sequences and triggers:

create sequence test_seq
start with 1
increment by 1
nomaxvalue;

create trigger test_trigger
before insert on test
for each row
begin
select test_seq.nextval into :new.id from dual;
end;
/

And MySQL is just an auto_increment keyword on the end of a column declaration:

CREATE TABLE animals (
id MEDIUMINT NOT NULL AUTO_INCREMENT,
name CHAR(30) NOT NULL,
PRIMARY KEY (id)
)
So I created a simple Autoincrement class:

/protected/migrations/Autoincrement.php

Which I included in the migration file for a schema.  This class will handle checking the driver and executing the appropriate sql.
Autoincrement::up('USERS', Yii::app()->db->driverName);
Autoincrement::down('USERS', Yii::app()->db->driverName);

What needs to be remembered here is to add a “sequenceName” to the model class, that is of the form “TABLENAME_SEQ”.

Using Facebook authentication with Yii (PHP)

Note: There many ways to implement this, but this seemed to make the most sense to me at the time.

A login can either come from someone clicking a login link and being sent to a login page or we can force the login and not allow guests.  The login/logout button is easy enough, just modify the distributed site controller’s login and logout “action” methods.  In order to force the login, the best wat to do this is to implement a behavior.  Please see Larry Ullman’s blog for more information on that.

Then we get to the IdentityFactory.  Yii has a nice configuration system in place for its components, so I did some Identity components extending the standard Yii UserIdentity that was included with the Yii distribution. I have the login entry point call on the identity factory which checks a param in the config and returns an instance of the appropriate object.

These objects are where all authentication type specifics happen.  Lets take a look at a simple flow diagram:

Facebook auth flow

So the IdentityFactory chooses which identity we’re using based on the config.  It will send the request over to the FacebookIdentity, which gets all the info needed for the UserIdentity class to query the database, update information there, and then set the user session.

The more interesting part is the connection between FacebookIdentity and the Facebook SDK.  For this I made use of a Yii extension.  I used yii-facebook-opengraph, which, while not the most mature of facebook connect extensions, is the most actively developed and the closest to functional.  (Last year Facebook made a huge change to their SDK which is not at all backwards compatible so they broke most extensions that exist and most developers did not make updates to their extensions.)  This extension only needed one method added to help deal with the Facebook problem with access tokens.


// had to add this function to deal with php's poor handling of expired access tokens
public function setAccessToken($access_token){
return $this->_getFacebook()->setAccessToken($access_token);
}

 

Problems with the Facebook PHP SDK

There were a few reasons I wanted to add Facebook authentication to the new iteration of Quizmo.
  • I wanted a second form of authentication to help in abstracting authentication.
  • I needed a second form of authentication because the intention is to open source this project, and I need to provide something other than iSites authentication if I want people to use it.
  • The process is very similar to Harvard’s PIN authentication, which I have done before, but probably won’t be necessary for Quizmo as it will be available through iSites.
  • “Everyone” uses Facebook so it’s probably something worth knowing.
The first minor problem was there is no proxy support.  The Facebook SDK takes you to a Facebook login page to authenticate, but before it sends you there, it checks if you’re already logged in.  This is done through a curl request.  The problem comes if you’re working in a development environment which doesn’t have a direct outlet to the internets.  The curl options are set as

public static $CURL_OPTS = array(
CURLOPT_CONNECTTIMEOUT => 10,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_TIMEOUT => 60,
CURLOPT_USERAGENT => 'facebook-php-3.1',
);

So this is stored as a static variable in the abstract BaseFacebook class – which is absurd because that means it can’t be overridden.  As such the following line has to be added to the BaseFacebook class:

CURLOPT_PROXY => "my.proxy.address:myport",

The larger problem with the Facebook PHP SDK is its inability to handle token errors.  The tokens are set server side when someone authenticates the first time.  These tokens then expire and the SDK is unable to handle errors that occur because of this.  The weirdest thing about this error is that the developers have a solution, they just expect people using their SDK to implement it on their end.  Regardless, their code did not work for me as advertised and had to be tweaked as such:
private function checkAccessToken(){

$app_id = ;
$app_secret = ;
$my_url = ;

// known valid access token stored in a database
$access_token = ->getAccessToken();

$code = @$_REQUEST["code"];

// If we get a code, it means that we have re-authed the user
//and can get a valid access_token.
if (isset($code)) {
$token_url="https://graph.facebook.com/oauth/access_token?client_id="
. $app_id . "&amp;amp;amp;amp;amp;redirect_uri=" . urlencode($my_url)
. "&amp;amp;amp;amp;amp;client_secret=" . $app_secret
. "&amp;amp;amp;amp;amp;code=" . $code . "&amp;amp;amp;amp;amp;display=popup";

// Now all file stream functions can use this context.
$response = $this->curl_get_file_contents($token_url);
$params = null;
parse_str($response, $params);
$access_token = @$params['access_token'];
}

// Attempt to query the graph:
$graph_url = "https://graph.facebook.com/me?"
. "access_token=" . $access_token;
$response = $this->curl_get_file_contents($graph_url);
$decoded_response = json_decode($response);

//Check for errors
if (@$decoded_response->error) {
// check to see if this is an oAuth error:
if ($decoded_response->error->type== "OAuthException") {
// Retrieving a valid access token.
$dialog_url= "https://www.facebook.com/dialog/oauth?"
. "client_id=" . $app_id
. "&amp;amp;amp;amp;amp;redirect_uri=" . urlencode($my_url);
error_log("top.location.href='" . $dialog_url);
}
else {
error_log("other error has happened");
}
}
else {
// success
//error_log("success" . $decoded_response->name);
//error_log($access_token);
}

return $access_token;

}