Doublehops

A new client whose job it is to audit water management systems approached me to build a complex document management system to automate an existing system that required a lot of manual labour. The audits of varying intervals result in written reports that needed to be kept for future reference, and notifications need to be sent when the system determines that a report is late. In point form, the major requirements of the new system were:

• Allow a new site (such as a building) to be added/updated with the following attributes:
  • List or auditing companies for given site
  • Report types and their required intervals
  • Email credentials
• Include an interface to create a report scanner – which will be used to determine the report type by scan a incoming report (pdf, xls, doc file) and scrape metadata such as report date and report type. This has to be done for each report type for each auditing company.
• Automatically download from the mail server any audits emailed from any number of reporters for the given site (eg. building)
• Scan the report using the report scanner, give it a filename that includes relevant metarandata and move to a CDN.
• Have a reporting page for each site to show the latest reports and show whether they we received on-time, late or overdue.
• Have another service hosting a Nextcloud service that would make available the reports for a user using their preferred Nextcloud client (web browser or mobile app). Therefore any facility manager could easily see and search for any reports for a location the were responsible for.
• Include a cronjob to look for any outstanding reports and send notifications to the facility manager for the given location.

Taking in all requirements, I determined AWS was the best platform to host the application as I could easily host the application and Nextcloud, use S3 for the CDN, use SNS for sending notifications and load balancers to place the servers behind.

Ansible was used to provision the local development environments as well as the EC2 instances on AWS for the application servers and Nextcloud servers which were all run on Debian.

I chose Yii2 framework to build the application for several reasons but mainly because I could easily scaffold each form and add validation and the business logic required. A custom component was written to retrieve new documents from the email server and scan them, and running the metadata through the scanner app. The documents were named according to their metadata and then moved across to an S3 bucket. If a document type was not found, then it was copied to a folder waiting to be fixed manually. A cronjob runs daily to scan the S3 folder for report types and check against the location’s report interval setting to determine if any reports were overdue and sent notifications if any were found.

The client is really happy with the application as it has taken away manual work as well as notify when reports are late which wasn’t available before.

I have been building an API with Yii2 for the last four months and and I’m really enjoying it. The developers of Yii have done a tremendous job putting together a framework that makes building applications both fun and fast. The API is to be used by iOS and Android apps as well as a web app (Angular JS) to facilitate phone calls and messages, among other things.

Many of Yii’s strengths come from the clever and well thought-out design that makes common tasks incredibly easy to implement and more complex tasks easy to build whilst keeping the code and structure of your application clean. Some of my favourite attributes to see are:

RESTful services

Building a RESTful service is incredibly quick to build by calling the inbuilt scaffolding tool. It will read the database table to build the model, and also views if you’re building a traditional website. Then with just a few changes in the controller you can start creating, reading, updating and deleting records through your browser.

Behaviours

Attaching a behaviour to a component or controller is similar to extending a class but gives you more control and can act on events. Yii2 has a collection of behaviours that are built in but allows you to create your own. Two built-in behaviors that I’m using are Authenticator and CORS.

Authenticator behaviour: This behaviour allows you to easily implement common authentication models such as HTTP Basic Auth and OAuth. But I was able to add a custom authentication module to the auth behaviour to meet our unique rrequirements.

CORS behaviour: CORS (Cross Origin Resource Sharing) is a standard that allows browsers to make Javascript requests to servers that the Javascript files themselves weren’t server from. This is generally not possible due to security issues. CORS can be quite complicated and can cause major headaches but Yii2 allows you to simply include the behaviour to a controller and the API will respond with the required headers to each CORS request from the browser.

Custom authentication

Yii2 has a great solution for Authentication and has the common ones, such as OAuth and Basic Http Auth built in. But it also quite simple to add your own authentication if need be. The requirements for us was to allow each user to connect to the API with multiple devices. Therefore each device is required to send its individual token for each request made to the API. Yii2 makes this quite trivial.

Testing

Yii makes it easy to use use test databases and environment variables which makes testing easy. My solution was to setup an alias domain (env.myapi.test) for each environment. When Yii receives a request on a domain ending with .test it will use test variables and database to run my tests and not pollute the dev or prod data with test data. Every API endpoint is tested and it’s very reassuring watching the tests that you have not broken any endpoints when new features are added.

I found enough time recently to finally look into Yii2. I decided that a good test project would be to build a crypto currency exchange tracker. It would download latest prices of all markets from both Cryptsy and Mintpal and then display the data in charts so I could quickly scan trends of all currencies.

Yii2 and it’s dependencies can be installed and managed through Composer which I enjoy. It prevents you from needing to keep any third party packages in version control and makes installs, upgrades and deployments easier. The Yii2 documentation is again great and the community is already solid. Any questions I had that were Yii2 specific were answered on the forum in good time. Some things are different and migrating projects from Yii 1.x to 2.x will take a lot of work. Yii2 uses namespaces and this means namespaces need to be declared at the top of views and other files that wasn’t previously necessary. Getting instances of records is slightly different and was changed several times during Yii2’s evolution. However this is stable now.

Many things are still the same. Migrations, scaffolding, commands and nearly everything else is the same. In my opinion, Yii2 is still the best PHP framework and I can’t wait to start a production project with it. Yii2 is still beta but the code base has mostly settled with only bug fixes remaining. My next task is to incorporate AngularJS into my Yii projects.

During the process of moving my websites from one host to another and from Apache to Nginx I came across an issue that had me frustrated yet intrigued. I have a Yii application that I use for my business that I ran over SSL. This was working fine under Apache and for the most part under Nginx. The problem would only come about in Nginx when a form was posted, the browser would be redirected from HTTPS to HTTP mode. Why was this?

After some Google searching, Yii core framework investigation and some trial and error I found that Yii relies on what I determined is a non standard HTTP header ‘HTTPS’ with a value of ‘on’. Nginx does not send this header and on a form post Yii would make the assumption that the browser was in non HTTPS mode.

The core Yii method that determined the protocol is:

public function getIsSecureConnection()
{
    return isset($_SERVER['HTTPS']) && !strcasecmp($_SERVER['HTTPS'],'on');
}

This can be overcome by adding a parameter into the Nginx virtual host record but I see this as a workaround and not a real solution. I thought a simple change from the non standard ‘HTTPS’ header to ‘SERVER_PORT’ (checking that it’s set to 443), which is sent by both Apache and Nginx would be a better solution. I have sent a bug report off to the Yii project with my suggestion so maybe we’ll see it in a future release.

Update: 18 October 2011
My suggested fix was added to the Yii code base but was reverted back out because a site running on port 443 does not necessarily mean it’s running over SSL. There is no standard header that browsers send when running over SSL so therefore a non-ideal solution must be used. I believe the best solution is to add the HTTPS header to the Nginx config as suggested.

Each month someone from the PHP Melbourne user group will present a PHP framework to the rest of the group. The November meeting gave me an opportunity to present the Yii PHP framework and why I think it’s a great platform to build PHP projects from.

During the presentation I described the main qualities I like about the framework and demonstrated how quick and easy it is to begin a new project.

I recommend for anyone looking for a new php framework to give Yii a try. It’s a great framework that allows rapid application development. The best qualities I find in the framework include:

• Componenet based
• MVC
• DOA/Active record
• Scaffolding
• Form validation/re-population
• Theming
• Authentication and role-based access control
• Strict OOP (PHP5 only to make full use of PHP object functionality)
• Great documentation
• Active community
• Example project

As with learning anything new there is always a learning curve. I suggest when starting out to try the Yii blog tutorial which takes you through installing a new project, building controllers and models from the database structure to extending Yii.