Leaving W3C

Two years ago I announced I was joining W3C as a full-time staff to work on the WebPlatform project.

A detail I didn’t share was that, like many of my W3C teammates, we are freelancers attached to one of the W3C host sites — mine was with MIT. Like any contract, it has an ending date and by tomorrow, mine will be over.

I’ve spent two amazing years improving the WebPlatform.org website. It was really a dream that came true.

I’ve worked on many projects such as improving the server deployment strategy in which we can now basically shut down every component of the site and rebuild from scratch only using source-controlled configuration management scripts.

One of the best things of being part of W3C, even though I was working most of the time not in a team, was that I had a great opportunity to work in collaboration with my wonderful colleagues, and my (now former) manager Doug Schepers.

It’s been a pleasure and a privilege to get to work with you all, and I won’t forget the great moments, the conversations, the travels, the challenges. But my time is up now, I got to hand my [ssh] keys.

I hope our paths will cross again.

The W3C Team taken in 2013 in Shenzhen, China.

Photo credits: Taken by Richard Ishida, during TPAC 2013 in Shenzhen, China.
(source: W3C team gallery).

Make Discourse “long polling” work behind Fastly

While working on deploying Discourse, I’ve came across a statement that took time for me to understand. Discourse has a subsystem that’s similar to WebSocket and Server Sent events which takes care to automatically update the page asynchronously. Note that this post is canonical version of my answer.

The confusing part was;

@sam
To server “long polling” requests from a different domain, set the Site Setting long polling base url to the origin server.
For example, if your CDN is pulling from “http://some-origin.com” be sure to plug in http://some-origin.com/ into the site setting. If you don’t your site will be broken.

This post is about clarifying why and how to work around this particular problem.

The confusing part is that if “http://some-origin.com/“. If you are behind Fastly, you have to use a CNAME entry and then you have to have a sub domain name and not the top level.

Background: In DNS, a top level domain name (i.e. “some-origin.com”) can only have A records. Since Fastly requires we use a CNAME entry, we have no choice but to use a sub domain name.

Let’s say that we will then use “http://discourse.some-origin.com/” to serve our Discourse forum so we can use Fastly.

Now there’s this thing called “long polling” which is basically an OPTION HTTP request with a long time before returning anything. If we use the Fastly or Varnish address, as Discourse would by default, Varnish will time out and “long polling” won’t work.

More background: Varnish has this option to bypass in known contexts through vcl_pipe which is roughly a raw TCP socket. But Fastly doesn’t offer it because of the size of their setup.

Proposed setup

Let’s enable long polling and expose our site under Fastly. We’ll need two names, one pointing to Fastly’s and the other to the IP addresses we give within the service dashboard.

  1. discourse.some-origin.com that’s our desired Discourse site domain name
  2. discoursepolling.some-origin.com (pick any name) that we’ll configure in Discourse to access directly to our public facing frontend web server

In my case, I generally have many web apps running that are only accessible from my internal network. I refer to them as “upstream”; the same term NGINX uses in their config. Since this number of web apps you would host on a site can fluctuate, you might still want the number public IP address to remain stable. That’s why I setup a NGINX server in front that proxies to internal web app server. I refer to them as “frontends”.

Let’s say you have two public facing frontends running NGINX.

Those would be the ones you setup in Fastly like this.

Fastly service configuration, at Hosts tab

Here we see two Backends in Fastly pannel at Configure -> Hosts.

Notice that in this example i’m using 443 port because my backends are configured to communicate between Fastly and my frontends through TLS. But you don’t need to.

Quoting again @sam;

@sam
To server “long polling” requests from a different domain, set the Site Setting long polling base url to the origin server.

Really means here is that we would have to put one of those IP addresses in Discourse settings.

What I’d recommend is to create a list of A entries for all your frontends.

In the end we need three things:

  1. What’s the public name that Fastly will serve
  2. Which IPs are the frontends
  3. Which hostname we want to use for long polling and we’ll add it to our VirtualHost

The zone file would look like this;

# The public facing URL
discourse.some-origin.com.  IN CNAME global.prod.fastly.net.

# The list of IP addresses you’d give to Fastly as origins/backends
frontends.some-origin.com.  IN A 8.8.8.113
frontends.some-origin.com.  IN A 8.8.8.115

# The long polling URL entry
discoursepolling.some-origin.com.  IN CNAME frontends.some-origin.com.

That way you can setup the “long polling base url” correctly without setting a single point of failure.

In Discourse admin, adjust long polling base url setting

Then, we can go in Discourse admin zone and adjust the “long polling base url” to our other domain name.

# /etc/nginx/sites-enabled/10-discourse

# Let’s redirect to SSL, in case somebody tries to access the direct IP with
# host header.
server {
    listen      80;
    server_name discoursepolling.some-origin.com discourse.some-origin.com;
    include     common_params;
    return      301 https://$server_name$request_uri;
}

server {
    listen      443 ssl;
    server_name discoursepolling.some-origin.com discourse.some-origin.com;
    # Rest of NGINX server block
    # Also, I would make a condition if we are in discoursepolling but not
    # under using anything specific to polling.
    # #TODO; find paths specific to polling
}

To see if it works; look at your web browser developer tool “Network inspector” for /poll calls on discoursepolling.some-origin.com, and see if you have 200 OK status code.

Use your web browser developer tools and inspect network traffic to see if requests made to discoursepolling worked

Note that this screenshot is showing webplatform.org but that’s beside the point i’m trying to illustrate.

Hope this helped.

Thoughts about learning in the web developer job, what managers might be missing

After reading an article titled What the ‘Learn to Code’ movement is forgetting: Existing developers, I couldn’t resist commenting. Sadly, I couldn’t add my own comment because their commenting is allowing only paying members so I am sharing it here.

Developers do not simply convert paragraphs of text requirements into “morse”. Obviously. Its a complex craft and like anything else, its all about people you are working with. There are passionate programmers and also the ones who only wants to do the bare minimum. In between, its all about leaving room for creativity.

What a programmer’s day looks like

Let’s remind ourselves what a programer is required to do has a lot of complexity and has to deal with legacy, unknowns, and delivery dates. Not everything can be foreseen: how the team members will work together, their capabilities, ego, and most importantly, whether or not each of them will be in “the zone” long enough during work hours.

The time they are at their optimal is what you want to have most from them, its that time where they can do amazing things, isn’t it. Being developer is about solving puzzles, review code or existing components they can leverage or would need to refactor. Being interrupted by numerous meetings to talk about something they are already trying to figure out doing isn’t helping them.

A good way to help them is to set in place asynchronous communication channels such as IRC, and code review practices. In some open source communities, merge to the master branch requires code review, a common practice is to have a bot to announce commits on an IRC channel. That’s the best of both worlds, you let developers be in their zone AND collaborate. As for paired-programming, they can just announce at the morning SCRUM meeting that they would like to spend time, or just ask for it on IRC.

Learning

As for learning, passionate developers already keep themselves in the loop of best practices and how to improve their craft. The experienced ones might even have ideas on how do things that you never thought of.

I’ve seen companies who allows participation to open source projects, and/or share their knowledge openly on sites such as Stack Overflow and documentation sites. This is another great way to make developers more engaged to their role, and stay in your company.

When I think of learning resources for web developers in general, there is an independent, and vendor neutral, resource where web developers can learn. The site is convened by the W3C, and sponsored by industry leaders to create just what we’ve been missing all along. The site is webplatform.org, and I feel like it’s not enough set forward by the industry. Maybe because its not complete, but its a wiki, anybody can participate.

Full disclosure; I work full time on the webplatform.org project, but note that regardless of my current position, I would participate to the site regardless of whether or not i’m hired to do so.

Photo credits: Anthony Catalano, Morse Code Straight Key J-38

Notes of my purchase experience with ASUS slim DVD-RW reader SDRW-08D2S-U

How do you feel when you purchase a tech product from a respected brand and it feels sketchy and you figure out, after receiving the desired piece of hardware that it doesn’t fit at all.

What got me even more disappointed is the fact I went through product reviews, read tech specs, shopped around for other products and decided to give a chance.

Before getting in the depth of the topic; I’d like to apologize for the tone of this message. My “gage of good will” (ref: Steve Krug ) is very low and I hope this gets somewhere so it can become an incentive to improve the site and product support medias.

NOTE I have (some) theoretical knowledge about User-Interaction design due to my numerous conversations on the topic with relatives and professional contacts who are in the field of software ergonomics. Its in the spirit to remove pain-points that i’m outlining them. Since I spent some time writing this essay, i decided to also share it on my site.

0. The product

img My original expectation wasn’t very big. I wanted a lightweight, small DVD recorder for my backups. You know, to store files you do not need often. I also wanted to encode what I’m burning on disc so if I ever have to let go of the disc, i can be reassured that the data is not accessible without cracking it.

So, what I was expecting:

  • Small
  • USB2 powered (only one cable)
  • Lightweight

My purchase was then a ASUS SDRW-08D2S-U

And supporting encryption got me close the deal. Sadly, ASUS didn’t do anything for Mac OS users.

The following are problems separated by scenarios I went through while getting acquainted with my new purchase.

1. Product registration

During product registration, it FORCED me to fill ALL fields, including my date of birth. If I do not want to give my underwear size besides my full name and region of the world I am at.

NOTE: After going to account, I could add what I wanted to disclose.

2. Searching for drivers

It says supports Mac OS right? After whipping the CD in my CD-Drive-less MacBook Pro I expected to see some (hopefully bloatware free) software to burn CDs and enjoy the CD crypto offering (more on this later). I found nothing.

I know Mac OS and the underlying BSD architecture has us no need to install any drivers. But since the package said it supported and provided software, I thought I could see if I install them. But found none.

After this, i went the asus.com. Its nice looking, marketing flash Pizzazzz. But not useful.

Its only after 10 minutes searching the site, only finding how to compare another product to buy and cannot find appropriate way to navigate to product, downloads/drivers. I got only promotional navigation. This makes the human who purchased something frustrated.

The funny thing is that I had to use a search engine that is NOT from asus.com (in useful results, again). Found plenty of other places to buy and then I found that page where I made this comment by using the comment form at the bottom.

The fact that the page had no images made me think of two things; one more detail that makes it look sketchy, and that maybe the page has outdated information and therefore would not apply to my case.

3. Disc Encryption

Now that I spent some more time explaining all this, I am very disappointed by the product because it sells crypto, but I guess only under Windows. My last resort will be to create MacOS native encoded DMG files, then burn them. No thanks to ASUS. :(

5. Y-shaped USB

Really?

That’s a first time. I understand the electrical needs and the fact that many laptop USB outlets doesn’t always feed with enough voltage/amps. I was surprised.

I saw NO reviews about this.

6. Reading the manual

img I saw NO reviews about this. The interesting part is that, unlike most people, I actually read the manual and it was a pain to figure out which boxes was in my own language.

It was basically me skipping each boxes, in case it was in either french or english. Just to realize after flipping the full “map” around that I just needed to plug the cable.

Good manuals doesn’t require long text. Have a look at how IKEA deals with multi-lingual instructions.

4. Conclusion

Needless to say that i’m disappointed by the purchase. But in the end, I have a not too big/clumsy, device that I can use to read and burn DVDs.

I really hope that ASUS gears toward improving their software products.

Image credits:
* IKEA instructions, by Mike Sacks
* Product picture is promotional material from ASUS website.

Recette du succès en quelques règles simples

Je copie pas souvent du contenu trouvé sur le web, mais cette publication est tellement bien faite que je n’ai pu me retenir!

Recette du succès.

  1. Si tu ne fais jamais d’erreurs, tu ne prends pas assez de risques
  2. Attends-toi à être patient pour obtenir quoi que ce soit qui a de la valeur
  3. Travaille très dur
  4. Recherche et demande des opportunités
  5. Réponds «oui et», à presque tout
    6 Assure toi de remplir tes promesses lorsque tu dis «oui» (!)
  6. Être «Occupé» est une décision en soi
  7. Ne censure pas tes rêves avant d’avoir vraiment tenté de les réaliser
  8. Apprends et construit à partir des rejets que tu as vécu
  9. Un réel échec arrive seulement si tu as l’a acceptée

J’ai trouvé la recette à partir de ce tweet, j’ai pris la liberté de le traduire et d’en conserver une copie sur mon blogue. Aussi j’ai ajouté une règle personnelle après celle du «oui».

I am joining W3C to work on the WebPlatform project!

webplatform.org

In a recent post, I was explaining that I resigned my position.

The new exciting project that I was referring to is the WebPlatform project at the W3C.

For people who do not know what the W3C is, it is the main international standards organization for the World Wide Web.

The WebPlatform project is a rapidly growing web development collaborative documentation website.

I would even call it the Web developer’s missing manual for web developers to build quality web applications using the latest techniques.

As for the sponsors, it is backed by well known companies, that we refer to as stewards, such as Adobe, Google, Mozilla, and Microsoft to name a few.

I will contribute to the site as a Developer Operations engineer. My work will be basically to enhance the features and manage the server infrastructure of the site as much as continuing what I already do as a hobbyist speaker.

Roughly, it consist of everything I always did or dreamed of:

  • Work on a flagship site with heavy traffic usage, with and for respected professionals around all the world
  • Use my favourite operating system and tools in System Administration
  • Technical liaison with Open-source communities
  • Develop and maintain the community platform, with web development, ensuring application performance, and implement continuous deployment
  • Speak and participate at conferences related to web standards
  • Contribute to moving the web forward, full-time (!)

If you are living in Montreal and you would like me to share with you on how to participate in this exciting project, just send me an e-mail (my first-name AT w3 dot org) and I’ll gladly answer.

I have just resigned from my new job to start on an exciting project!

In the last months, many things happened to me.

First, I worked for almost two years in a contract at Ericsson, but it ended. Therefore, I started searching for a new job. I had some offers and I had a hard time to pick one.

However, after all, I have just received a new offer to work for the W3C; something I cannot that I could not turn down.

I am very excited to have this opportunity. I would even say more: it is a honour.

I will explain more about it in another moment.

For now, I would just like to thank TEKsystems‘ team and to mention the many good things about working there:

TEKsystems

TEKsystems is a company based in the USA and their main solution development center is located in Montreal Downtown. Most of the projects are web based business applications serving many customers in the united states.

Among the things I enjoyed most were:

  • The management team uses Agile methodologies wherever applicable;
  • The leadership is strong, and the team leads are skillful;
  • The multiple coding technologies in one place: Java, PHP, Frontend, iOS, Android, .NET,
  • The multidisciplinary team: Architecture, Business Analysis, User experience;
  • The members of the team, at the end of the project, get shuffled in the office to start a new one;
  • The internal structure to create mini-projects so we can learn and contribute from other employes;
  • The fact that the office is located in Montreal Downtown, and right beside McGill University where I am starting a Continuing Studies Courses certificate

I worked there for the last month and a half, and it is amazingly well organized company. Based on my own experience in different work contexts, I can say that working with them for web development projects was enjoyable.

If it wasn’t because of my new opportunity, I would have stayed there for a long while.

I am joining W3C to work on the WebPlatform project!

In a recent post, I was explaining that I resigned my position.

The new exciting project that I was referring to is the WebPlatform project at the W3C.

For people who do not know what the W3C is, it is the main international standards organization for the World Wide Web.

The WebPlatform project is a rapidly growing web development collaborative documentation website.

I would even call it the Web developer’s missing manual for web developers to build quality web applications using the latest techniques.

Continue reading

Why would I NOT use my own framework and how I sell usage of Symfony2 and other current PHP 5.3+ goodies to my client

Sometimes, in a mailing list I am following and in real life. People are sometimes explaining their choices in technology to create an e-commerce site using a CMS (“Content Management system”) or their own “homemade” framework. As if there is no good PHP ones out there nowadays.

A CMS is a web application to manage content and when specialized requirements (as in managing other things than pages of content) we have to extend outside of the CMS part. A framework is thus a good friend.

In either case. My opinion is to use the right tool for the right task.

I wrote on the subject because I recently had to defend my technological choice of Symfony2 at my (very-large) client. You can see on these two posts.

I tend to believe that everybody who can program can work with a well architectured CMS/framework whatever software architecture patterns it is following. The priority is that we, programmer, fills the client’s requirements and use what is existing to make us efficient.

Creating a “just for this project”-“super custom”-framework and saying it is using [a buzzword-compliant] architecture principle, doesn’t PROVE IT well made.

Don’t get me wrong. It is not that creating a framework is bad. It is quite an educative hobby. Creating a framework requires a lot of work, dedication and ideally a quantity of skilled people to work to improve, review, test and document. A full time job by itself alone.

My feeling is that the client probably did not asked to create a “framework”.

My problem is more precisely about using it as production-code for their business application, make them pay for it AND what they explicitly asked for. Should they only pay for the features they asked for?

But if you used a well known, documented, framework with an active community. You can use it as a good selling argument in front of the client. It even reassures them that other people can take on your work.

So a question lingers in my mind. If one has the skillset to create his own framework. Why doesn’t he contribute to an existing one?

Personally, all my choices follows the NIH approach. NIH as in “not invented here” see Derek Strobe’s blog post about NIH, a much inspiring read.

The overall idea is that Instead of charging for my time implementing dependency injection, i’ll use an existing one.

Maturity

Talking about maturity. Symfony2 is 2 years old already (compared to Zf2 few month of age).

If it is not only about maturity. Think that the PSR (levels 0, 12)  recommendation is written by a group community leaders who decided to standardize the way to structure libraries so they are inter-operable, PHP FIG was created for this.

I think everything changed since the PHP-FIG group created the PSR* standard and the use of composer to manage dependencies. CMS and other framework projects have to follow the lead, otherwise they may get left behind.

CMSes

I have the feeling that someday CMS will be only a GUI, with full edition capability (see Mercury Editor, or Createjs) using HTML5’s  ContentEditable property. And persisting inside a Storage engine known as “Content Repository”.

This trend is also what some calls “Decoupled Content Management” (also explained here), a VERY promising concept that will give some more chances for us to actually re-use code.

The prospect may even make us be able to really create easily semantic-friendly pages. Created by editors. Regarding semantic web, you should look at “VIE” over here.

The trend is more than mere theory. Many project spawned already. Most notably the Symfony-CMF, Midgard and even known CMS-es such as Drupal and Joomla are currently re-evaluating their projects and following the trend.

Dependencies

A common problem is how can I make sure the project uses proper libraries and a compatible version number.

Many stacks has their own implementations of it. It comes as a simple file stating which version you want, and it takes care of getting the proper version from any configured package we may think of.

The concept is now new and many web development stack has their own implementations. Have a look at package.json (for Javascript), Bundler (for Ruby).

PHP has it’s own now, it’s called Composer, here is how you would declare your dependencies:

{
   "require": {
       "myown/vendor": "master",
       "external/lib": "3.1.*",
       "external2/template": "master"
   }
 }

See: Introduction to composer using Packagist

Add to that any cache management, deployment configuration and other build librariescompilation and protection you may wish to use.

All in all

Nowadays. If you are still:

  • Manually using require_once(…)
  • Adding database queries in your view (WordPress? x_X)
  • Using loops and query hidden in a … templating engine (ExpressionEngine “loops” x_X)

You may need to see the reference I made in this post.

My argument goes as following, mostly for what my (our) clients pays us for:

  • Learn and make good use of framework that has a reputation, good architecture, modular, inter-operable with other, in PHP is not a chimera anymore.
  • NIH please
  • Build functionality for the client who pays you. not maintain the serializer service mechanism… framework do that stuff. client do not care about it
  • SOLID principles?
  • Scaling? (only one database? what if I need to go deploy on a PaaS solution?)

Réflexion a voix-haute: La réalité frustrante des développeurs dans l’industrie du web

Je suis conscient que cette situation n’arrive pas qu’au développeurs web. Je raconte ici ce que j’ai vécu pour illustrer comment je trouve que c’est un non-sens. Je doute qu’il y ait réellement de solution autre que de s’armer d’argumentaires et la pratique.

Parmi les efforts que je trouve admirable est celui fait par l’Alliance Numérique avec son guide des bonnes pratiques.

Résumé d’une histoire de réalisation web

Une personne du monde marketing a une idée. Elle veut que ce soit super beau. Toutes les vues sont faites sur 40 pages faites par Photoshop et doivent être respectés au pixel parfait sur IE6 à Firefox 3. Il y a 70 pages de descriptions de comment javascript doit faire sortir des étoiles partout. Ensuite, pendant la réalisation, la compagnie a pas de budget, généralement elle fait couper les coins ronds aux endroits cruciaux et le site sera bon seulement pour changer ton adresse et inutilisable.

What is Cloud computing when it is related to web application

During the discussion, the contributor persisted on knowing what would be considered and thresholds to use some kind of push-button-scaling.

Knowing his context, a unzipped install CMS with a buch of plugins I felt the urge to explain that there is not always need to get a bigger server capacity. Here is an overview of what I mean when I talk about cloud computing and continuous integration.

The E-Mail

Let’s talk about cloud! 

I mean in the web application hosting realm. Not the storage (Google Drive, Dropbox) or software as a service (Salesforce, Basecamp).

Let’s talk about a use case before and my own experience.

My former company Evocatio Solutions technologiques manage a pretty large site at the domain uda.ca.

The use-case on my recent experience

This is a complete business management web application that manages an union who represents french speaking artists in north america (mostly residents of Canada). We built a complete web application that manages many aspects an artist needs to represent themselves and be found. A big part of it is a 140 tables worth of artist description listing details as small a hair length and types of musical instruments to voice tones. It also manages renewal, communication with agencies, portfolios, and management of contracts with managers and more.

Not to forget the very heavy databases queries we generate to search, for example: <example>An asian woman with white hair playing yuku lélé who can pilot helicopter AND ride motorcycle …</example>

Yes. Database queries get very big, very quickly. Not only in the search engine I described, but through all the features.

That, to my opinion, is heavy. Also considering that that Artist’s Union has several thousand members.

This information is on top of my head, please do not take this into real numbers, I did not look the latest deployment needs.  But for the server side, it only uses a simple Virtual machine with 4Gb of RAM give or take.

That is my point about expanding hosting without optimizing stuff around.

What your web application has to consider then

Amazon and other Cloud service is about mostly about automated server deployment.

But the powerful offering of “scale tour application” with computing cubes that automatically scales requires more than just nodes.

It requires the code (here again) to support:

  • multiple databases hosts and types support (Cassandra, Solr, MySQL) specialized for the type of data to store
  • User upload files replication
  • Database/Keystore (CouchDB, Mongo)

All spanable on multiple hosts by a mere change of one configuration file.

The code itself should:

  • Be deployable by a simple phing/ant/nant task
  • Hosted on a NAS mount that you could create an other machine and use when time of computing need happens

All this (for some parts) is what is called Continuous integration (Wikipedia) some deployment strategies (also here and this blog post too), and most of the time. It’s not just the continuity and automation that matters, but the underlying deployment mechanism can be provided by third parties, like Heroku and many others.

Some steps you can look for if you feel your web application is slow

It all started by a discussion thread in a mailing list. A guy who developed a shopping cart and payment gateway using a CMS.

My first reflex, before thinking of scaling the server I thought I should give some pointers on things that can hog the site, before going to think to scaling solutions.

That was all before continuing talking on the cloud that I answered later that I answered later on that blog post.

The thread started as follows:

> (…) I have a Magento modified into a e-commerce site (…) that
> me and my client feels slow, my client has asked about Amazon hosting. They
> do everything else, CDN, the works, shouldn’t their hosting
> be superior?
>
> What would be worth for a test drive, I’d say, if you think
> your site’s performance issues can be addressed by
> throwing CPU, memory, storage, etc (…)

My answer

I doubt that you need bigger hosting for a e-commerce site.

Not for the “first  thing to improve performance” action point though.

Unless your site has to provide (real) HEAVY traffic. Non stop.

It is most likely something somewhere down the execution of the web application that requires to be looked at.

Performance slowing factors

Some common explanations for slow execution time could be basically due because of one or many of the following:

  1. Network latendy
  2. Process communication problem (connection, zombie process, etc)
  3. Application architecture
  4. Hardware/Software performance

Now talking about application architecture.  This one can be a real can of worms!

Some Application architecture bottlenecks

I currently seriously doubt the order here should matter. But this is the ones that pop into my mind at first.

  1. Web service/database queries/files access across network … packetloss could also be a cause
  2. Database queries processing that could require some well picked indexes
  3. Heavy queries and frequent read write or some sleep() hidden here and there to wait other result set
  4. No http/view caching
  5. No code caching/precompiled code at all (can be config, whatever that can be pre-compiled and served into basic arrays of calculated data frequently used)
  6. No memcached/keystore service
  7. No read-only data store

An analogy

So. To my opinion, if you are using a “unpack to install” web based software such as WordPress, then add plusings without testing and looking if all of them works well together.

You are likely to be trying to make a Cheetah kitten into a humanoid Android.

As in,  you can install a lot of metal patches. Doesn’t mean it will have a full AI system and be self sustainable.

That is to illustrate, what it is lie, you should look to alternatives. At least with something closer to a droid :)

Seriously enough.

My professionnal recommendation would be to work with each “application architecture buttolenecks” proposal list before “going cloud”

My answer to People asking whether they should or not use a framework on their programming language

I think that whatever we do when it is related to programming, the main objective is that the piece of functionality is to be used.   Unless of what you are doing is a tool for developers, or a programming language.  To my opinion, you should be concentrated in the functionality itself. Wathever the reason of your application, it is to be used. Otherwise it’s waste. Isn’t it?

This is not because it is NOT a good thing to do.  Just that there is a lot of good framework and libraries out there.  Why diluting the efforts when you could contribute and make something good with your own goodness.

Here is what I answered

The question

Am I the only one not beaing lover of PHP frameworks, asked in a PHP Developers group on LinkedIn.

My answer:

Flame wars. again.

I read an article by Derek Strobe on his blog, discussing  the Not Invented Here mentality and I actually think the same.

Why not focussing on: getting the best you can get and/or not-reinvent-the-weel.

What is the important, build the developer tool, or build the fatures the client pays us for.

Framework or whatever programmer use, should only help us for at least those two:

  1. Have a standard that you are not the only one to know (documentation, and or can suck. sorry.).
  2. Do the features the client wants, with the thing that makes you shine most.
  3. Educate the client for the features, not the tool. You are the expert, that’s why they hire us.

For me, framework or not. Building a framework for myself, it would suck. I prefer to abide by the framework structure to it’s best recommended. And contribute to the documentation (or features) in return.

Using a framework, it’s name, version or not use one. It all comes back to the structure the code you have to work with (from either source, legacy, framework, current)

If you have a method with more than 10 lines… it’s procedural in a class. Like halloween.

A last thing about: I did my framework, and I still think it’s the best.

Nobody will make me really believe that it’s better than a more than dozen people contributing to quality and psr-0 compliant classes re-usable libraries.

That’s what happened with Symfony 2 for me.

I dare you to to find a fully fledged structure **in PHP** that already implements: AOP, Event listeners, Namespaces (like Java import), Closures, Dependency Injection, and close to the root of the language.

One last thing, if you want to distribute:

You should also look at Composer. A way to distribute your package.

Project manifest: Content management publishing platform to implement Accessibility, Semantic markup, and ease web publishing

This is my own manifest on how I think today’s CMS should be architectured. I need to find a proper name on the idea and/or possibly join one of the project that exists. If you understand french, I created the equivalent post on that page.

Layers:

  1. Data extraction layer, takes from…  xml, rss, github, wordpress database, drupal database, REST service and pass to layer 2
  2. Data filtering, normalization and serialization. Providing adapters for normalization, caching, and expose a gateway for third layer
  3. Content transportation layer of normalized/serialized content transport mechanism
  4. (need to coin term here)  “Container”  providing:
    1. Placeholders for content coming from layer 3
    2. Automated creation of HTML blocks and patterns following semantic markup and accessibility principles
    3. Logicless templating for easy usage for integrators who do not need to learn programming

Advantages on that approach

I truly think that with such mechanism, we could adapt content in a semantic manner and be able to abstract content edition, theme.

Much is here to say about it. I am leaving this as it is now and I’ll update relations and ideas as I spend time on it. Contact me if you have questions.

Nouveau projet: Refonte de mon site en conservant WordPress comme back-office, mais Symfony2/Doctrine2/Twig pour générer les vues

J’ai décidé de refaire mon site web. J’aime bien utiliser le backend de WordPress mais je n’aime pas utiliser ce qui y est fourni coté code pour faire le “frontend” d’un site web.

Il s’agit d’une première élaboration brute des concepts qui motivent mon choix d’implémentation.

Je suis en plein effort d’élaboration des requis et si vous voulez voir le pendant rédigé en anglais, j’ai publié ici.

But ultime des fonctionnalités

  • Utiliser l’administration de WordPress pour gerer le contenu
  • Avoir la latitude de Symfony2/Twig/Doctrine2 pour gerer le contenu e le code
  • Aggreger le contenu provenant de d’autres sources (marqué favoris de Google reader, Tweets, status divers de d’autres services: forrst, github, geeklist, etc)
  • Creer une libraire de macros Twig qui genere du HTML selon les formats de Schema.org
  • Contribuer a un outil qui pourraitservir aux webmestres qui desirent utiliser WordPress mais qui veulent ce que Twig/Doctrine offre.

Fonctionnalités prévues

  • Extraire, purifier/harmoniser le html, et convertir les billets de blog en format markdown (version cachée)
  • Utilisation des billets format markdown comme cache
  • Conversion markdown a html pour consultation
  • Commit-push des billets format markdown dans repository git configurable
  • Processus Migration données WordPress (galleries, commentaires) vers version markdown
  • Suite de macros reutilisable au format schema.org qui n’est pas lié a la base de donné
  • Conversion de format markdown avec annotations des medias, publication, lien canonical, etc)

Lignes guides d’un CMS idéal

  • Le contenu doit être libéré de son système de conteneur.
  • Le contenu doit être le plus vanille possible, le plus près possible du texte brut.
  • annotation dans le contenu (titre, lien, gras, italique) doit être écrit dans un format texte SEULEMENT (markdown, reStructured, Textile). Permet l’édition sans avoir a maîtriser toutes les nuances du balisage (markup html) qui est requis pour l’accessibilité, ou la sémantique
  • Les éléments de contenus sont des blocs de HTML distincts du contenur qui l’affiche. Dans le sens que la page et le style doit être aussi flexible que possible et que l’intégrateur ne soit pas emêtré dans des syntaxes abstraites
  • Le système doit permettre d’utiliser plusieurs fournisseurs de contenu (une base de donnée WordPress, Drupal, un repository Git, etc)
  • Le système doit permettre de comprendre l’entité de contenu et générer automatiquement le bloc HTML sémantique approprié pour être affiché dans son conteneur

Le Code

J’utilise présentement mon implémentation en lisant la base de donnée WordPress de mon site, et utiliser Symfony2/Doctrine2/Twig pour le “frontend”. On peut voir la différence de vitesse d’exécution et le look entre beta.renoirboulannger.com, à l’inverse de la version hébergée sur le domaine non “beta”.

Fork me on GitHub

Résumé d’une discussion et notes sur ce que j’aurai aimé comprendre plus rapidement lors de mon apprentissage Symfony2 et Doctrine2

Durant les derniers mois j’ai lu et relu la documentation de Symfony2 et voici quelques éléments qui ont attiré mon attention et que j’aimerai partager avec vous.

L’un des concepts architecturaux clé faisant que Symfony2 se démarque des autres frameworks PHP est l’usage des concepts de la programmation orienté aspect (AOP).

L’idée directrice de la programmation orienté aspect est d’éviter au maximum dépendances fonctionnelles communes (appelé dépendances transversales). Ces fonctionnalités fréquentes comme l’accès a la base de donnée, par exemple. La manière de séparer ces besoins est de notamment utiliser les commentaires des classes et d’y ajouter des fonctionnalités avant ou apreès les méthodes. C’est ce qu’on appelle des Intercepteurs et des Pointcut.

Ce type d’architecture demande une capacité de compilation de code supporté par le langage de programmation.

Nous savons que PHP n’est pas un langage compilé. Il y a des alternatives, mais ce n’est pas comme ce qui peut être fait avec AspectJ et Spring AOP, pour Java, et ce qu’il existe pour le C#.

Pourtant, depuis peu, avec l’avènement de l’implémentation des normes PSR-x, et des librairies de génération de code. C’est maintenant possible.

Beaucoup de changements sont a venir dans le monde des CMS et pour beaucoup d’autres projets.

Le mouvement est tellement fort que les communautés de d’autres projets libres s’impliquent: Ce projet s’appelle le “PHP Framework Interoperability group” Il y a notamment le projet Magento, Drupal, Joomla!, MidgardTypo3 (Flow3), voir la liste des groupes et du draft sur github.

Donc, ces choses que j’ai beaucoup apprécié lors de ma découverte de Symfony2.

Conversion des entités et éviter requêtes à la base de donnée

Comme la documentation le décrit, le travail d’une méthode de contrôleur est de convertir la requête qui lui a été demandée d’exécuter (via le routing). L’utilisation des patterns AOP vient permettre de maigrir énormément.

Au lieu d’utiliser manuellement les Exception, le render, et les appels à Doctrine faits “à bras” dans le corps de la méthode comme suit.

namespace PSS\Bundle\BlogBundle\Controller;
 
use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Sensio\Bundle\FrameworkExtraBundle\Configuration\Route;
 
/**
 * Blog Controller
 **/
class BlogController extends Controller
{
 
    /**
     * @Route("/{slug}")
     */
    public function showAction($slug)
    {
        $entityManager = $this-&gt;get('doctrine.orm.entity_manager');
        try {
            $post = $entityManager
                -&gt;getRepository('PSS\Bundle\BlogBundle\Entity\Post')
                -&gt;findPublishedPostOrPage($slug);
        } catch (\Doctrine\ORM\NoResultException $exception) {
            throw new \Symfony\Component\HttpKernel\Exception\NotFoundHttpException('Page Not Found');
 
        }
 
        return $this-&gt;render('PSSBlogBundle:Blog:show.html.twig', array('post' =&gt; $post));
    }
}

On peut sauver des lignes de code et utiliser les @annotation pour définir le Template qui sera utilisé, va aller chercher l’entrée dans la base de donnée pour nous (via le @ParamConverter), retourner une Exception si elle existe pas.

Vous pouvez voir plus en détails l’implémentation dans Dans mon exemple pris de mon fork du bundle PSSBlogBundle qui sert a lire la base de donnée WordPress de mon site, et utiliser Symfony2/Doctrine2/Twig pour le devant de mon site (beta.renoirboulannger.com, à l’inverse de la version hébergée sur le domaine non “beta”).

&lt; ?php
namespace PSS\Bundle\BlogBundle\Controller;
 
// Symfony/Doctrine internal
use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Sensio\Bundle\FrameworkExtraBundle\Configuration\Route;
use Sensio\Bundle\FrameworkExtraBundle\Configuration\Template;
use Symfony\Component\HttpFoundation\Request;
 
// Entities
use PSS\Bundle\BlogBundle\Entity\Post;
use PSS\Bundle\BlogBundle\Entity\Term;
 
/**
 * Base Blog Controller
 *
 * @Route("/blog")
 */
class BlogController extends Controller
{
    /**
     * @Route("/{year}/{month}/{slug}")
     * @Template()
     */
    public function showAction(Post $post, Request $request)
    {
        return array(
            'post' =&gt; $post-&gt;onlyIfPublished(),
            'comment_form' =&gt; $this-&gt;comment($request,$post)
        );
    }
}

Ok, toutes les lignes de codes sauvés sont devenues des lignes use au haut de la page. Mais tout ceci a le mérite d’être très explicite. La magie se passe principalement dans l’annotation @Route qui fait la relation avec l’entité Post et va ajuster la requête de recherche dans le PostRepository et limiter sa recherche au colones slug, year, et month. Ces colones peuvent ne pas exister mais simplement être des méthodes. C’est la magie de Doctrine2 et totalement hors du sujet de ce billet.

Autres annotations d’intérêt

@Route

Cette annotation permet de contrôler les requis que l’adresse doit remplir pour être utilisée. Elle nous permet de spécifier quelle propriétée est liée a l’entitée déclarée dans la déclaration de la méthode du contrôleur.

@Template

Le @Template annotation sur un controleur sert a remplacer les appels a render avec nom de template et va suivre la convention du nom logique, exemple: AcmeBlogBundle:Blog:show qui va aller chercher le fichier de template dans le dossier Acme/BlogBundle/Resources/Blog/show.html.twig.

Il est bien de remarquer ici que par défaut l’engin de templating est Twig et la sortie est HTML. Mais tout peut être configuré. Puis j’aimerai ajouter l’emphase que le système peut permettre de fournir les requis de sortie sous format JSON et nous n’aurions qu’a créer un fichier au même nom mais insérer json à la place de html, exemple:

Donc, plus tard dans le projet, il sera faccile d’ajuster l’attribut “_format” puis dire si header is xmlhttprequest accept`applcation/json, utilise le template au meme nom mais …json.twig et avoir les memes controlleurs repondre html ET JSON :) la methode su controlleur qui a un @Template s’attend a cwe que tu donne un array cle valeur que tu peut utiliser dans ton template.

@Observe

Celle ci est ma découverte de la semaine avec @Inject. Au lieu de déclarer dans un fichier services.yml, il est possible de déclarer directement dans la méthode de la classe l’événement qu’elle est addressée à traiter.

On peut voir dans cette réponse sur StackOverflow comment on peut ajuster le TimeZone sans avoir a modifier d’autres fichiers dans le projet.

Mes autres références

Mon fil delicious delicious avec tags: symfony2 et very-helpful vous donnera ma sélection de références que j’ai apprécié.

Pour comprendre plus

Quelques autres références et billets qui m’ont inspiré et aidé a comprendre.

Des billets inspirants:

Outils à considérer

Trying to find templating engine library of markup generating schema.org/RDFa content

I am currently working on a project that I am going to publish on github.

It’s roughly a library of templating engine macros and blocks to send collections of data (similar to a JSON format) and it builds the HTML favouring HTML+RDFa with abstraction of the CSS schema for implementation using this “framework of things”.

My question is mostly if any of you guys had been aware of already made such libraries.

What I am working on:

The main idea is to be able to exchange content, take from external source, cache in a “serialized” but text only (readable) way, then generate the html in the shema you want.

If you want to see in detail the big picture I want to attain using what I am searching, I suggest you look on the project manifest here.

Where I am at:

My current implementation is made in Twig using Symfony2/Doctrine2 with a WordPress database.

For the moment I have my site beta.renoirboulanger.com/ (not stable yet) which implements partly the first and the third part, you can see the code of the first part in this github repo branch, you can see how I am architecturing in this Wiki page.

So far, for the third part, I have only a few patterns using schema.org/BlogPosting and schema.org/UserComments and I am abstracting the css vocabulary out of it and making sure the templating library is as DRY as possible.

When I will have more than an incomplete BlogPosting+UserComments macros I will open the project on github.

Your input

As I said, I am mostly wondering if you have knowledge of any such implementation of set of macros that represents html in micro formats, be it in Python Django templating language, Mustach, Hogan, or anything.

Thanks!

Les diapositives de ma présentation au HTML5Mtl “Comment entretenir et utiliser une architecture modulaire et réutilisable CSS” est publié

Vous pouvez les trouver ici: Comment entretenir et utiliser une architecture modulaire et réutilisable CSS.

La présentation a eu lieu pendant la rencontre d’Avril du HTML5Mtl. Le groupe HTML5Mtl est un sous-projet du W3Québec pour promouvoir la nouvelle norme du W3c. Le HTML5.

Mon expérience sur le sujet est assez forte car depuis 2010 je ne fais que ça, du HTML5. En fait, j’y décrit un sommaire de mes recherches faites dans la dernière année chez Evocatio.

Sommaire de la présentatin

Avec le temps, j’ai envie d’être optimal sur tout ce qui touche le projet, et rendre réutilisable:

  • structurer le code de Front-end
  • structurer le code Back-end
  • le processus de déploiement

Avec tout ça, il y a des avantages qui me semblent très intéressants:

  • le processus de conception (wireframes en html) permet de valider toute modification CSS si on brise quelque chose d’autre ailleurs dans l’application
  • les programmeurs n’ont pas a inventer de markup html, ils pigent dans le “style-guide”
  • au fil du temps, on se retrouve avec le même HTML, et du CSS spécifiquee au projet

Expérience passée

C’était ma 2ème occasion de présenter devant des gens. La première a été durant un devCamp de Microsoft sur le sujet “How to manage big scale HTML/CSS project“. Le même sujet mais avec beaucoup plus de matière.

Finalement

Je crois que je devrais ajouter du contenu et améliorer sa structure. Je compte ajuster ma présentation avec:

  • Exemples concrêts pour chaque premeière introduction d’un concept
  • Référence cohérente comme hyperlien pour chaque mention
  • Ajuster le “ce que j’assume” au début de la présentation, et m’assurer que l’audience sait de quoi je parle
  • Donner plus de raisons sur ce que je considère une mauvaise pratique

Vous voulez que je présente?

Je suis disponible pour faire présentation dans votre école, université, groupe. Laissez-moi le savoir.

Feuille de style pour imprimante pour les pages de documentation de Doctrine2 et Symfony2

Je suis en train d’étudier activement Symfony2 et Doctrine2 car j’ai fait le saut du coté développement applicatif a temps plein.

Mon apprentissage va très bien et j’ai appris beaucoup sur les meilleurs pratiques et je ne me vois plus dutout faire de php sans le Dependency Injection, les pratiques de namespacing et fonctions lambda de PHP 5.3.

Alors, pour étudier, j’ai décidé d’imprimer les pages de la documentation. Malheureusement il y a beaucoup de perte d’espace pour du contenu de navigation et temporel qui se ramasse dans mes PDF.

J’ai converti en PDF plus de cent documents, puis, finalement j’ai repassé les imprimer en ajustant certaines règles CSS.

Je partage ici le CSS que j’ai élaboré pour pouvoir imprimer les documents sans avoir de perte d’espace pour le lecteur tablette.

Comment utiliser

Pour appliquer mon impression, j’ai simplement crée ces blocs CSS, puis inséré ces derniers dans chaque document via l’inspecteur de Google Chrome, puis imprimé.

J’ai aussi contacté les auteurss des sites respectifs pour leur proposer d’y insérer mes règles.

Je pense ne pas être le seul a avoir eu besoin d’imprimer leur documentation.

f

Documentation Doctrine2

Je me suis basé sur les pages de cette section.

J’ai aussi ajusté le fait que les blocs de configuration alternatifs (yml, php, xml) soient tous visible. En imprimé si on veut comparer la syntaxe, on n’a pas javascript pour activer les tabs.

@media print {
  #footer,
  .footer_popout,
  #nav.cls,
  #content .sphinxsidebar,
  #content .related,
  body a[href~="github"] {
     display:none !important;
  }
  #content .bodywrapper {
      margin:0;
  }
  #content .bodywrapper .body {
      max-width:initial;
  }
  #content {
      font-size:120%;
  }
  #content div.body h1,
  #content div.body h2,
  #content div.body h3,
  #content div.body h4,
  #content div.body h5,
  #content div.body h6 {
      background:none;
  }
}

Documentation Symfony2

Sur toutes les sous pages de la documentation.

@media print {
  #content_wrapper .box_download.clear_fix,
  #content_wrapper .main_menu.clear_fix,
  #content_wrapper .main_content .column_01,
  #content_wrapper .main_content .column_02 .navigation,
  #content_wrapper .box_relative_content,
  #content_wrapper .footer .box_menu_footer.clear_fix,
  #content_wrapper .footer h2 img,
  #sln {
      display:none !important;
  }
  #content_wrapper .main_content .column_02 {
      width: 100%;
      padding:0;
      font-size:120%;
  }
  #content_wrapper .footer {
      background-color: #FFF;
  }
  #content_wrapper .footer h2 {
      color: #444;
  }
  #content_wrapper .footer h2:before {
      content: "Symfony";
  }
  #content_wrapper .highlight-jinja {
      font-weight:bold;
      font-size:130%;
      width: 100% !important;
  }
  #content_wrapper .highlight-jinja .hilight {
      background:none;
      border:1px solid #444;
  }
  #content_wrapper .highlight-jinja .hilight:before {
      content: "Code block";
  }
  #content_wrapper .configuration-block.jsactive.clearfix {
 
  }
  #content_wrapper .configuration-block.jsactive ul {
        height:initial !important;
  }
  #content_wrapper .configuration-block.jsactive ul.simple {
        overflow:hidden;
        height:initial !important;
  }
  #content_wrapper div.jsactive div div,
  #content_wrapper div.jsactive div {
        position: relative !important;
  }
  #content_wrapper .configuration-block.jsactive ul.simple li {
        float: none;
  }
  #content_wrapper .configuration-block.jsactive ul.simple li &gt; div{
        display: block !important;
        width: 100% !important;
  }
}

Comment rapidement accéder aux logs d’un environnement de production symfony 1.x en moins de tapes clavier posible

Est-ce que ça vous est arrivé de vouloir voir les logs de votre application symfony 1.4 en production pour un problème et qu’aucune info n’est disponible pour savoir quel est le problème.

Ça m’est arrivé une fois de trop cet après midi. Ma solution est simple, utiliser le concept du yaml de symfony qui accepte du PHP, puis de faire UNE switch qu’on peut “flipper” pour pouvoir consulter le log.

Sans interruptions de serveur, ni pertes dangeureuses de performances **

Comment j’ai fait

J’ai ajouté a mon projet un simple bloc conditionnel qui va comme suit.

1. Fichier factories.yml

Ajouter cette condition dans le bloc prod:

prod:
< ?php 
/**
 * En cas de besoin debug, flipper 0/1 pour pouvoir accéder aux logs.
 *
 * 0 = Pas de debug
 *
 * Ne pas oublier de faire un ./symfony cc :)
 **/
if(0){  
echo'
 
  logging_enabled: true
  logger:
    class: sfAggregateLogger
    param:
      level: debug
      loggers:
        sf_file_degug:
          class: sfFileLogger
          param:
            level: debug
            file: %SF_LOG_DIR%%SF_APP%%SF_ENVIRONMENT%.log
 
';
} else {
echo'
 
  logger:
    class:   sfNoLogger
    param:
      level:   err
      loggers: ~
 
';
} ?>

2. Nettoyer la cache

Tout simplement.

./symfony cc

3. Suivre le log

tail -f log/frontend_prod.log

Bonsoir!!