Using Templates with Perl Applications
part of Perl for the Web
After Perl code can be embedded in HTML pages, the next logical step is to make the process of writing Web-centric programs easier. The embedding environments listed in Chapter 12, "Environments for Reducing Development Time," have the raw capabilities that enable Web-centric programming to be automated to some degree. One way to accomplish this is by incorporating templates into Perl Web applications.
The idea behind a template is to separate formatting information and program structure to keep the development of each part separate. The template itself is a document that contains formatting instructions that are used by the program to create a final product–in this case, an HTML page. It is usually hoped that the separation can be accomplished in a generalized form to enable one template to be used for all programs in a given group.
Generalized templating also gives designers the freedom to create a template without the need to know anything about the programs that will use it. Thus, a template language usually is created to enable program features to be indicated in an abstract form. For instance, a template language designed for a news site could have defined placeholders for the title and body of a story, as well as for story images and related links. Because any number of stories share these basic elements, it's possible to write a template that could be used across all of them. In addition, because the code that displays the stories might change (for example, drawing story data from an Extensible Markup Language [XML] file rather than from a database), it's better to create templates with abstracted sections than with specific code.
At its simplest, a templating system involves two files, the template and the program to which it is applied. In practice, there are likely many template files and even more application files using the templates. The programs usually handle their interaction with the template files on their own using a template library or custom code. In addition, some template systems use a third type of file, a program that manages the relationship between templates and templated applications. This template manager might be incorporated into the Perl embedding environment itself, or it might be written as a Web application preprocessor to be executed before embedded applications.
Template HTML and Graphic Editors
The primary goal of a template system is to give site designers a representative document they can create and edit with ordinary HTML tools. This means that the template document should be as close to standard HTML as possible, even when it contains the indicators that indicate where application segments fill in parts of the template. As a result, template HTML can't be radically different from standard HTML because it might not render correctly in its raw form within graphic HTML editing software.
On the other hand, templates are useful only as long as the tools used by site designers don't interfere with the template language itself. Sometimes, the abstracted code included in a template document can be removed or altered by the graphic HTML editors used by designers. If this happens, unexpected results can occur when using the template document in a Web application. Luckily, this can be avoided somewhat by using template syntax that is unlikely to be modified by common HTML editors. In addition, preferences in the editing programs themselves can sometimes be changed to accommodate the templating language.
Of course, there's only so far that the software can go to make a designer's life easier. It's also necessary to develop good guidelines for designers to follow so that they will maintain good template structure. If designers have a good grasp of the template language and how it looks and is treated in their favorite HTML editing software, it's much more likely that Web applications will end up looking as expected.
HTML That Doesn't Interfere
A good way to accommodate graphic editors with a template system is to incorporate the template information into valid HTML segments that are either represented in a visual way or ignored completely. The goal behind this idea is to affect the formatting of the template HTML in one of two ways:
- Not at all. Template code is either rendered using a nonobtrusive symbol or coloring on the page or completely ignored by the page previewer. This usually occurs when template code uses comment-style code or custom HTML tags.
- Exactly as the final Web application would. This could be as simple as displaying variable names for information that will be filled in by the application or using entire blocks of representative data that is excised later. This includes text-like placeholders for text and image-like placeholders for images as well as entire sections that might be filled in by the Web application.
With HTML 4.0, a number of new tags and structures were introduced to make this process much easier. The <span> tag, for instance, provides a way to declare the style of a specific area of an HTML document. In a template context, it also can be used to denote the location of a section to be filled in by a Web application. The following template code segment could be used to fill in a news story:
Listing 16.
...
<span class="newsblock">
<h2><span name="app.headline">Headline</span></h2>
<p><span name="app.abstract">This is where the story would be.</span></p>
</span>
...
The template contains two sections to be removed by the templating application and replaced by generated text. The first section denoted by the <span> tag, app.headline, is removed and replaced by headline text generated by the application. The second section, app.abstract, is removed and replaced by another text segment generated by the application. The resultant text might look like the following:
Listing 16.
...
<span class="newsblock">
<h2>Damian Conway Wins Honors</h2>
<p>Conway won the White Camel award Tuesday for most awards received.</p>
</span>
...
Note that the <span> tag is used for two purposes within this template section. It is used to provide placeholders for the replaced text, but it's also used within its original HTML context to denote a style class called newsblock. Differentiation between the two uses is possible by using the name attribute on template sections and the class attribute on standard HTML spans. This is possible because most HTML editing software treats both types of <span> tags as valid; it simply ignores the ones without class attributes. The Web application then can pick out only those <span> tags with the name attribute to use as template sections.
Another common way to set template information apart is to use comment-style sections that are ignored by even the oldest browsers and graphic editors. These sections can be extracted by the template processing code in a fashion similar to <span> sections. The comment-style version of the previous example might look like the following:
Listing 16.
...
<span class="newsblock">
<h2><!-- start app.headline -->Headline<!-- end app.headline --></h2>
<p><!-- start app.abstract -->This is where the story would be.<!-- end app.abstract --></p>
</span>
...
A third way to indicate template sections is to use Perlish variables in a style similar to double-quoted strings. This kind of template is sometimes much more efficient to process because it can be dealt with in a native Perl fashion. Perl-style variables also are more likely to fit into HTML templates in which tag-style sections wouldn't fit, such as the following example:
Listing 16.
...
<span class="newsblock">
<h2>$headline</h2>
<img src="$story_image_url" align="right" />
<p>$story_text</p>
</span>
...
Here, the variable $story_image_url is used to denote the image URL text to be filled in by the Web application. This section is used to fill in part of an HTML tag rather than a plain text section or a complete HTML tag. Therefore, HTML editors have less trouble rendering it than they would have rendering an invalid HTML tag with another HTML-like tag nested inside it. In practice, a combination of variable-like section identifiers and tag-like section identifiers combines the best aspects of both systems.
Effects of Graphic Editors on Templates
No matter what the templating system is, it's possible that an unaware HTML editing program will remove or modify the template information. Many graphic editors take a "smart" approach to HTML similar to the approach Word takes to word processing–they make changes to the document based on internal rules instead of expecting the user to make them manually. This usually saves time and produces better output when dealing with pure HTML, but even minor changes to templating code can upset the result of a Web application.
For instance, the template document in Listing 13.2 later in this chapter would be modified automatically by graphic HTML editors like Netscape Composer because the file doesn't follow traditional HTML format. The file has the following general format:
Listing 16.
<tag name="template" accepts="title, section" body="1">
<output>
<html>
<head>
<title>VeloMeter - $title</title>
</head>
<body bgcolor="white">
...
</body>
</html>
</output>
</tag>
Because the file doesn't start with <html> and end with </html>, Composer assumes that the user has made a mistake. As a result, it moves the <html> and <head> elements of the page so that they start before the <tag> and <output> elements, assuming that the latter actually are supposed to surround the <body> element instead. The result ends up looking something like this:
Listing 16.
<html>
<head>
<title>VeloMeter - $title</title>
</head>
<tag name="template" accepts="title, section" body="1">
<output>
<body bgcolor="white">
...
</body>
</output>
</tag>
</html>
Unfortunately, this causes the template code to behave differently than expected. This particular template requires all elements of the resultant page to be contained within the <tag> element, for instance. Thus, it ends up excluding the <html> and <head> elements from the template as it's processed. Even worse, in this case, all text outside the <tag> element is printed as soon as the file is included. Therefore, the resulting output would both start and end the <html> element before the <body> element was reached.
In addition, some graphic HTML editors see template tags that they don't understand as extraneous formatting information, and they remove them entirely. Others automatically modify the template information to suit the page design as it exists. This is more likely to happen when repurposing standard HTML tags, such as <span>, as mentioned in the previous chapter. This also is more likely to happen when the editing application takes a more active role in creating the formatting, styles, and accompanying HTML of a document.
Even in the case of HTML comment syntax, graphic editors might automatically remove the comment as superfluous. Because there is no visual evidence that the tag has been removed, it's sometimes difficult to tell if this has even happened without viewing the source of the page every so often to check. On the other hand, some editors display commented code as an obtrusive icon that disrupts the flow of the page and encourages manual deleting. The icon sometimes can be a good indicator of where template sections are located, but designers with little knowledge of the visual effect of template codes still might remove the codes inadvertently when polishing the page.
The Best Defense: Social Engineering
In many cases, the only way to ensure that template information won't be excluded or modified by a graphic editor is to raise awareness of the template syntax among site designers. The tools being used to edit HTML are known best by the designers themselves. Therefore, it makes sense to foster an awareness of the general format of template code and the specific effects that might be seen in HTML editors. Of course, these same designers also are the people using the template language to indicate Web application sections within a template, so it's not too much to ask for a general knowledge of the language and its effects.
This is another reason to keep the template language simple and understandable. The more complex a templating language is, the more likely it is that designers will have difficulty learning all its possible permutations. If the language is too difficult to figure out, designers have a hard time even figuring out the relationship between the template and the output of the Web application. With a simpler set of codes and a more uniform method for denoting template sections, designers are more likely to absorb the full range of identifiers and determine how to best use them within standard HTML tools.
The complexity of template interactions also gives weight to the idea of a staging area for site designers to use when testing templates against live information. There's no better test of a template's interaction with Web applications than actually using it with an application in the correct environment. The staging area can be tailored to the templating needs with a set of sample pages generated using the new template and representative site data. However, the data and applications only need to be representative. It shouldn't be necessary to keep a complete copy of the live site up to date for staging use. Formatting can be checked against similar pages just as easily, and mirroring a live site might be more trouble than a staging area would justify.
Applying Templates Across a Site
The core reason for developing a template is to apply a uniform look and feel across an entire site or group of sites. Developing the template is the first step in this direction, but it's also necessary to modify other pages and applications on the site to adopt the template as well. A site template won't help uniformity much if the component applications and pages don't reflect changes made in the template document. As a result, it's necessary to set up a general way for all site documents to access the same template, no matter how it changes.
Templates can be applied across a site in a number of ways. The main difference between these methods is the complexity of the site and the corresponding complexity of the template system used. Simple sites usually need only one template document, which then can be referred to directly in all other documents and applications on the site. More complex sites–those with sections that look and feel different from each other– usually can be served by a group of similar template documents in a central location with a consistent naming scheme or a template manager providing access. Very complex sites–ones with issues such as user access control and personalized content–can be served by a dynamic template that changes depending on the application from which it's being called.
No matter how templates are implemented for a site, some thought should be given to future applications of the templates. Will applications a few years down the road be able to take advantage of the same templates designed today? Conversely, will applications written today be able to use templates that are designed for a radically different site in the future? If not, is there a way that the old templates can be adapted for the new applications, or vice versa? These questions should be kept in mind when designing template languages or putting template systems into place.
Single-Template Case
For many sites, a single site template suffices for all pages on the site, both dynamic and static. This template should be stored in a single file that is updated by the site designer directly. This avoids any derivative work caused by the artificial need for site administrators to modify the template for use with applications. Keeping a single copy also provides a central place for essential edits to be made by all site personnel. Thus, site designers have a current snapshot of the site design with which to work, no matter what changes have been made.
Of course, a staging area for template design might also be a good idea so that the effects of site modifications aren't instant and irretrievable. If that is the case, the same rules apply to the staging area copy of the template that would apply to a single copy of the template. The production copy then can be taken as a regular snapshot of the staging area template with no changes made directly to the production copy. Even more complex systems of distributed design can be used–including CVS and other content management systems–but the core idea remains the same: Use a single central copy (or repository) that is accessible by all site personnel.
The first aspect of a single template to consider is where to put the template so that it is accessible to all pages and applications on the site. This location needn't be in the site document tree because the template is likely to be processed by a Web application that has a longer reach into the Web server's file system. However, the template should be in an area from which the Web server user–the user the Web server is running as–has permission to read. In addition, if a Web application is used to create and manage the template files, the Web server user also should have write and create access to the template file and directory where it resides. (Create access is necessary because some template managers remove old templates before creating new ones.) It's also advisable that the template location be easy to remember and easy to type–the combination saves headaches down the line when page creators and Web application programmers need to reference the template in their files. For instance, a file path such as /web/templates/page.txt is reasonably resistant to typos and misspelling and easy enough for site designers to memorize.
After a suitable template location is determined, pages and applications on the site can be modified to use the template. For static pages, this might involve translating pure HTML pages into a format that can be handled by a template manager or embedded Perl processor. It also might involve changing the extension or content type of the static HTML pages so that they are automatically recognized and processed by the template manager application. However, in some cases, it's possible to define a preprocessor to add the template to the static files without modifying their content types or adding code to the pages themselves. Either way, modifications to all the pages and applications should have to occur only once to bring them into the template system. After that occurs, changes should be made solely within the template system itself.
When applying a template to site elements, care should be taken to leave no part of the site out of the core template system. Some applications might present problems when adding template processing code, but modifying the application for templating is worth any effort required. It might be tempting to simply copy the template into the application and adapt it for use within the program, but the simplicity of such an approach is deceptive. Any process that is required when translating a template for use with an odd application has to be repeated each time the template changes, which could be every week or every day, depending on the activity of site designers. Eventually, either the work required to continually update the template copy outstrips the work required to add templating, or the template copy stored in the application falls out of date and site unity suffers as a result. It is more robust in the long run to adapt all applications to use the central template from the start.
Multiple Templates
Sites with a more complex structure might need multiple templates to provide a different style for each section of the site. As an example, a site might need multiple templates if the home page of the site and other overview pages have a fundamentally different structure than pages that display a specific bit of data–a news story or a product description, for example. The need for multiple templates also can occur when a site uses pop-up windows for some types of pages–a shipping table displayed when a customer is using an e-commerce application, for instance. Multiple templates also can reduce the complexity of any individual template by enabling more variation in page design with less variation within the template itself.
A group of template files can be set up in a fashion similar to the single template case. In fact, a single template can be viewed as a special case of a multiple-template system. A template location should be determined and a template-naming scheme defined. In this case, the naming scheme also should involve a descriptive name that indicates the site section to which the template corresponds. Then, the templates themselves can be added using the defined naming scheme. Template code then should be added to every static and dynamic page on the site, with special attention paid to generalizing the template code so that new templates can be used as they appear, without modifying the code again.
After multiple templates are defined, there should be some way to distinguish between templates within applications and static files. The simplest way to distinguish between templates in most template systems is by specifying the filename directly within the code that calls the template. If this isn't possible within the bounds of the template system, it's also possible to specify the template file as an attribute of the template being called. For instance, a Web application being used in the forum section of a Web site might call a template file called forum.txt. The translation between section names and filenames can be handled by the template manager application. A concern here is the use of templates within preprocessor code. If a template location is hard-coded into the preprocessor program, that single template is used for all static pages, no matter which section of the site they are part of. One solution to this situation is to define site sections in terms of their URL locations so that the preprocessor can call different templates accordingly. For instance, every file in the /forum/ subdirectory of the Web document directory might use the forum.txt template file. This kind of inherent template specification might create limitations on the relation between the structure of a site and its file locations. However, many site designers already use site directories to organize the content of a site so that the change won't necessarily be a burden.
Dynamic Templates
For sites with structure that varies based on user preferences and permissions, a dynamic template enables page structure to be generated interactively. This comes in handy mostly in situations in which site content has to change based directly on the user that is accessing it. For instance, a site governed by user access control lists might show different content to different users depending on their access levels and areas of interest. A site also might generate page structure based on themes and choices specified by user preferences. In any case, the structure of the page is no longer a simple matter of the designer's wish, but more a matter of the combined choices and authorization of the user making the request.
Simple dynamic templates are made possible by embedding Perl within the template itself–assuming that the template manager enables embedded Perl. Sections of the template can be filled in by executing code based on the incoming request, just as in the Web application itself. For instance, conditional code can be added to a site menu that displays only certain menu choices to certain users. Other aspects of a dynamic template can be retrieved from a database or processed from an XML data file. The possibilities are as wide as with any Web application, but the result is open to all pages across a site. Because the template can be made available to a site preprocessor, a dynamic template can be used by static HTML pages and dynamic Web applications.
More robust dynamic templates can be created by developing a Perl module that provides methods to create page structure. Because the pages calling the template usually are embedding Perl into HTML, calling object methods from a template module can be just as simple as calling other kinds of template functions. A dedicated template module enables more control to be exercised over fundamental aspects of the Web application interface, including access control, session management, and user preferences. Module-based templates also enable nonembedded Perl code to access and use templates. This overcomes a hurdle that most simple template systems have trouble addressing: if the template document is written in an embedded-Perl style, programs written in a Common Gateway Interface (CGI) style might not have access to it. Any Perl program can access a Perl module template, so all programs benefit equally.
Of course, creating templates dynamically can invalidate one strength of template design–the ability to edit the templates in standard HTML editors. Because any Perl code is likely to throw off a graphic HTML editor, this is true regardless of whether the template is an HTML page with a little embedded Perl code or a full-blown Perl module. Luckily, the solution here is the same as the original template solution–separate the representative HTML out from the dynamic template as well. At this point, the resulting HTML is more a convenient construct than a working template for the look and feel of the completed site, but even a construct can provide enough information for site designers to create a coherent look and feel across the site.
A Simple Template Using Perl Server Pages (PSP)
Returning to the simple template case, it might be helpful to see an example of a template system fully realized. The system developed in this section is reduced to the simplest case possible: one template and one application using it. The system can be extended easily to use multiple templates and many templated applications, but the principles for that case are identical to this one.
I've chosen PSP as the environment for this template system simply because it's the environment with which I'm most familiar. A similar system could be created using HTML::Mason or a dozen other environments. PSP was chosen also because it's the server environment for VeloMeter.com, a handy site to which I have access and over which I have editorial control. PSP also has the basic requirements for a simple templating system built into its abstracted language–I can create a template tag within PSP with a minimum of extraneous work. This kind of reasoning is important when choosing an environment that enables templates to be designed easily.
In addition to the template system created here–or any custom template system created for a specific site–a number of existing template systems exist that work perfectly well for the purposes of a simple site like this. Those systems are covered later in this chapter, in the "Template Modules on CPAN" section.
Using the <template> Tag
The core of the template system developed here is a PSP tag called <template>. This tag performs the work of joining the output of a Web application to the site template. A few additional parameters are specified to provide a more integrated page style, but the core purpose of the <template> tag simply is to wrap a Web application in a predefined template shell. Listing 13.1 shows an example of this kind of templated application.
Listing 16.1 VeloMeter Page with Template (register.psp)
01 <include file="$ENV{DOCUMENT_ROOT}/page.psp" />
02 <template title="Load Testing Forum - Register" section="forum">
03
04 <p>Registration with the VeloMeter forum isn't necessary to post or view messages, but it does allow others to recognize messages posted by you as yours alone. As a result, we've tried to make registration as painless as possible.</p>
05
06 <p>We also respect your privacy. Your email address won't be posted on the site or used for any marketing purposes. The email is only required in order to provide a basic check of your identity.</p>
07
08 <form action="do_register.pperl" method="post">
09 <output>
10 <p style="color: red;font-weight: bold">$QUERY{error}</p>
11
12 <p>Your name:<br />
13 <input type="text" name="fullname" size="30" maxlength="50" value="$QUERY{fullname}" /></p>
14
15 <p>Email address:<br />
16 <input type="text" name="email" size="40" maxlength="200" value="$QUERY{email}" /></p>
17
18 <p>Username/Password:<br />
19 <input type="text" name="username" size="20" value="$QUERY{username}" />
20 <input type="password" name="password" size="20" value="$QUERY{password}" />
21 </p>
22
23 <p><input type="submit" value="Register" /></p>
24
25 <p>Once you register, you will recieve an email with confirmation instructions. You will be able to post messages immediately after confirmation.</p>
26
27 </output>
28 </form>
29
30 </template>
One thing that might be noticed about this page right away is its striking similarity to a simple HTML page. In fact, the page itself is close enough to pure HTML that it would be rendered just fine by itself in a browser, as shown in Figure 13.1.
***INSERT Figure 13.113hpp01SC***crop
Figure 13.1
This templated application can be viewed directly in a browser.
There are a few differences in the page due to the templating environment, though. Line 01 of Listing 13.1 contains an <include> tag that loads the template from its assigned location. In this case, the template is stored in a file called page.psp, which is located in the document root directory of the Web server (as specified in the DOCUMENT_ROOT environment variable), but it also might be found in an absolute directory, which contains only templates. In addition, different templates can be used in different applications by changing the file referenced in the <include> tag because the syntax of the rest of the file can stay the same.
The next alteration to the templated page is the <template> tag itself. This element encompasses the entire page to be displayed. It starts on line 02 with the first tag after the <include> tag and ends on line 34 at the end of the file. The contents within the tag are considered the body of the page to be inserted into the template, as specified in the next section of this chapter, "The <tag> Tag and page.psp Template." The <template> tag also has two possible attributes, title and section. The title attribute simply specifies the title of this page to be displayed in the appropriate parts of the template. The section attribute isn't actually used by the template example in the next section, but it might be used in the future to alter a dynamic template based on the section specified. Extra attributes, such as section, are good to add as soon as they are conceived to avoid editing the page again after the template is given additional functionality.
Other PSP tags are used in Listing 13.2, but they all are part of the Web application itself, not the template system. The end result of the <template> tag is a page that merges the template included by line 01 with the contents of the <template> tag. When combined with the template defined in the next section, the resulting page looks similar to Figure 13.2 in a Web browser.
***INSERT Figure 13.213hpp02SC***crop
Figure 13.2
The result after the template is applied.
The difference between the two pages is subtle, but the template provided a definite look and feel to the Web application page. The contents of the page are centered within a defined area on a white background, and a logo and navigation for the site have been added before the beginning of the page. In addition, the title provided to the <template> tag now is listed both as the HTML page title (as seen in the title bar of the browser window) and within the page as a more obvious heading.
The <tag> Tag and page.psp Template
The template used by Listing 13.1 has to define the <template> tag as well as provide the template HTML to complete the page. An example of a template that does both is shown in Listing 13.2, which provides some simple structure to the page and defines basic navigation as well. Note again that this page has to be viewable as HTML in a standard Web browser or in graphic HTML tools. As a result, extra care has been taken to make sure that the template logic doesn't interfere with the basic structure of the HTML in the template.
Listing 16.2 Template Definition (page.psp)
01 <tag name="template" accepts="title, section" body="1">
02 <output>
03 <html>
04 <head>
05 <title>VeloMeter - $title</title>
06 </head>
07
08 <body bgcolor="white">
09 <table width="80%" align="Center" cellspacing="0" cellpadding="0" border="0">
10 <tr>
11 <td colspan="3"><a href="/"><img src="/images/logo-velometer-200.png" alt="VeloMeter" border="0" /=""></a></td>
12 </tr>
13 <tr>
14 <td colspan="3" align="Right"><font face="Arial, Helvetica" size="2"><a href="/download.psp">download</a> . <a href="/features.psp">features</a> . <a href="/screenshots.psp">
15 screen shots</a> . <a href="/forum/">forum</a></font></td>
16 </tr>
17 <tr>
18 <td colspan="3" bgcolor="#333399"><font face="Helvetica, Arial, sans-serif"><img src="/images/dot-black.gif" width="2" height="2" border="0" /=""></font></td>
19 </tr>
20 <tr>
21 <td valign="top" colspan="3">
22 <table width="100%" cellpadding="10" cellspacing="0" border="0">
23 <tr>
24 <td>
25 <h2>$title</h2>
26 $body
27 </td>
28 </tr>
29
30 </table>
31 </td>
32 </tr>
33 </table>
34
35 <p align="center"><font face="Arial, Helvetica" size="1">©2001
36 <a href="http://www.velocigen.com/">VelociGen Inc</a></font></p>
37 </body>
38 </html>
39 </output>
40 </tag>
In PSP, a custom HTML tag is defined by using another custom tag, called <tag>. This tag can contain any valid PSP page content, which then is executed after the tag is used. In Listing 13.2, the <tag> tag is used in line 01 to define a tag called template by specifying the name of the tag in the name attribute of the <tag> tag. This name could be any valid XML string, but a good rule of thumb is to avoid names already used by HTML elements. Another attribute of the <tag> tag is accepts, which defines a list of variables that the tag defines in terms of specified attributes. In this case, the <template> tag accepts attributes called title and section that are assigned to variables with the same names. These variables are lexically scoped to their assigned tag, so multiple tags can use the same variable/attribute name without causing overlap or unusual behavior. The last attribute of <tag> is body, which indicates that the contents of the <template> element should be accessible within a special variable called $body. The <tag> tag ends on line 42, the last line of the template.
The variables defined within the scope of the <template> tag– $title, $body, and $section–can be called directly within the template text by using the <output> tag in lines 02 and 41. Everything within <output> then is treated like a double-quoted string in Perl. Lines 05 and 27 use the $title variable to display the specified title, for instance, and line 28 places the $body variable in context. That single line represents the entire result of any Web application or static file that invokes the <template> tag, just as the <template> tag itself represents this template in those files. These two constructs provide the abstracted interface between templates and Web applications that then can be extended across the site.
Enhancing Templates
A system like this could support multiple templates of the same format by simply varying the name of the file being included with the initial <include> tag. For instance, a different template could be used for each section of the site. The syntax of the <template> tag would stay exactly the same because the format of the tag being defined wouldn't vary at all.
Additionally, templates could incorporate other pages with headlines, news, or other section filler. This works particularly well when adding dynamic listings to a template that are common to the entire site but still generated by Perl code. An <include> tag could be used to load these sections into the template, further abstracting them away from the formatting of the page. This is similar to the component model used by HTML::Mason, which could be used to provide a modular template in the same fashion.
This kind of template file also can support dynamic features that change the format of the page based on attributes passed along with the <template> tag. The section attribute, for example, could be used to choose navigation features. This could be incorporated by adding a conditional using either Perl code or PSP tags. Additional dynamic page elements could be added as needed, but most of them would compromise the ability of site designers to view the page in an HTML editor.
Automating Template Updates
The ability to use templates is one thing, but it's not worth much without giving designers a way to update the site template automatically without needing to involve technical staff. Site designers need a way to update the look of a site in real time, even if they are only updating a staging site that is separate from the live site. It's also helpful to provide a Web interface to site updates because secure Web access is common to many platforms and requires a minimum of special software. In contrast, secure FTP or shell access requires software that is unlikely to be found by default on systems used by Web designers.
The simplest way to provide Web-based access to template updates is by creating a Web application to accept and track template uploads. This application can be written using the same framework that is used by all other Web applications on the site. The application even can use the template itself to provide a uniform interface with little extra work. A simple template management application is illustrated in Listing 13.3[nd]Listing 13.6 in this section.
A Template Upload Application
Listing 13.3 is a simple upload form that uses HTTP upload to provide a common interface to transferring files onto the server. The listing uses the same template as Listing 13.1, so the resulting page has the same look and feel as the rest of the site.
Listing 16.3 Template Upload in PSP (upload.psp)
01 <include file="$ENV{DOCUMENT_ROOT}/page.psp" />
02 <template title="Upload a new template">
03
04 <p>This page allows you to send a new template to the server.
05 Choose a file to send using this form, then click "Send file"
06 to check the integrity of the file and assign it a name using
07 the next page.</p>
08
09 <form action="do_upload.pperl" method="post" enctype="multipart/form-data">
10 <output>
11 <p style="color: red;font-weight: bold">$QUERY{error}</p>
12
13 <p>Template file to upload:<br />
14 <input name="template" type="file" /></p>
15
16 <p><input type="submit" value="Send file" /></p>
17
18 <p>Once the template is received, it will be previewed using a
19 test page. If the page loads correctly, use the form shown to
20 complete the template installation. If the page does not display
21 correctly, return to this page and upload the corrected file.</p>
22
23 </output>
24 </form>
25
26 </template>
One important feature of this Web application to notice from the start is the lack of Perl code in Listing 13.3, which simply displays the form and passes the result to another file. The idea here is to separate the display application from the application that does work. A PSP page handles the front end, while the persistent Perl page handles the back end. When used as a general principle, this method avoids some of the most common–and most dangerous–pitfalls of Web application design. Specifically, it avoids unusual effects if a particular page is reloaded, printed, or opened in a new window. If the page was performing work as well as displaying HTML, these actions would cause it to perform the work again. The results of this duplication range from additional load on the server to duplicate entries made in backend databases for orders, messages, and other submitted items. Because the page is only displaying a result instead of acting on server systems, however, it can be reloaded any number of times and provide the exact same visual result. That result is seen in Figure 13.3.
***INSERT Figure 13.313hpp03SC***crop
Figure 13.3
Template upload page.
The important parts of Listing 13.3 are contained in lines 09 and 14. Line 09 starts the <form> element, which points to the backend file do_upload.pperl and declares the encoding type to be multipart/form-data. This means that the resulting request can contain an uploaded file as well as other form variables. Line 09 follows through on that promise by using an <input> tag of type file. This results in the text box pictured in Figure 13.3 with the "Browse..." button next to it. A file path can be typed into the text box manually, or the button can be used to invoke a file selection dialog box with which the user can fill the text box automatically. The file specified in this box is sent to the backend application, along with its filename and other form variables.
Accepting File Uploads
After a file is uploaded to the server, it's necessary to retrieve the file and store it. Listing 13.4 provides a backend application that performs this task by saving the file to a temporary location and forwarding the user on to a page that displays a confirmation page.
Listing 16.4 Template Upload Back-End (do_upload.pperl)
01 use 5.6.0;
02 use strict;
03 use warnings;
04
05 use CGI ();
06
07 my $q = CGI->new();
08
09 # get the uploaded file
10 my $fh = $q->upload('template');
11
12 # if no file is uploaded,
13 unless ($fh) {error("No file was sent. Please select a file to upload.")};
14
15 # place the uploaded file in a temporary directory
16 open(TEST, ">$ENV{DOCUMENT_ROOT}/templates/test.psp")
17 or error("Can't write temporary file: $!");
18 while (<$fh>)
19 {
20 print TEST;
21 }
22 close TEST;
23
24 # redirect the user to the naming page
25 print qq{Location: set_name.psp\n\n};
26
27 sub error ($)
28 {
29 my $error = shift;
30 $error =~ s/\s/+/g;
31
32 # redirect the user to the upload page with an error
33 print qq{Location: upload.psp?error=$error\n\n};
34 exit;
35 }
Listing 13.4 is written in a persistent Perl style, which is similar to the CGI style, except that it assumes it is running in a persistent environment. Lines 01[nd]03 initialize the environment to ensure that persistent-compatible code is being written. Line 05 invokes the CGI.pm module–CGI.pm provides methods for reading uploaded files. Line 07 creates the CGI object $q, and line 10 assigns the $fh file handle to the uploaded file provided by the upload method of the CGI object. Line 13 checks whether a file has actually been uploaded and sends an error to be displayed in the upload page, if necessary.
After the file has been received, lines 16[nd]22 open a temporary location for the uploaded file and write it to disk. This location should be standard enough to enable the test page (as outlined in the next section of this chapter) to access it consistently each time. For this example, a fixed filename is used to store all test pages. Some systems might want to store each uploaded template in a separate file to provide accountability and backup.
After the file is uploaded and saved, line 25 redirects the user to a page that tests the new template and enables a permanent place to be assigned. An error subroutine is defined in lines 27[nd]35 to provide a generic way to pass errors back to the original upload page for display and reentry.
Checking the New Template
The best way to test a new template is to use it in a sample page. Because a successful template needs to be assigned a name on the server, it makes sense to make the test page a form that accepts the template's name as well. Listing 13.5 is an example of a test page that also serves as a completion form.
Listing 16.5 Template Preview Page (set_name.psp)
01 <include file="$ENV{DOCUMENT_ROOT}/templates/test.psp" />
02 <template title="Name this template">
03
04 <p>Check this template for validity, then choose a name
05 for the template and click "Install template."</p>
06
07 <form action="do_set_name.pperl" method="post">
08 <output>
09 <p style="color: red;font-weight: bold">$QUERY{error}</p>
10
11 <p>Template name:<br />
12 <input name="template" type="text" size="30" value="$QUERY{template}" /></p>
13
14 <p><input type="submit" value="Install template" /></p>
15
16
17 </output>
18 </form>
19
20 </template>
Listing 13.5 is a form that is similar to the upload form in Listing 13.3. The main difference in this listing is the template used to display this page–it should be the same one that was just saved by the backend application in Listing 13.4. Line 01 of Listing 13.5 loads the new template and its associated <template> tag; then it applies the template to the form HTML. The result, assuming all goes correctly, should look somewhat similar to Figure 13.4.
***INSERT Figure 13.413hpp04SC***crop
Figure 13.4
Template confirmation form.
The page in Figure 13.4 is shown using a template that differs slightly from the template in Listing 13.2. This is done to show how differences appear in the confirmation form. This page serves as a simple staging area for template design. Templates can be uploaded here repeatedly to check their performance and appearance and then saved to the live server when they are approved. If the template doesn't work for some reason or causes an error on the server, that also appears on this page. If another template needs to be uploaded, the user can access the upload form by returning to the previous page.
After it has been determined that the new template is satisfactory, the form can be used to determine the name assigned to the new template. In this case, the name is kept simple, but any naming convention can be used. Figure 13.4 also shows the error result of some simple checking performed by the naming backend application. Template names are restricted in this case to a single string consisting of letters, numbers, or underscores.
New Template Notification
If the new template is satisfactory, it is saved to the specified filename. Additionally, it might be wise to notify site personnel that the template has changed. Listing 13.6 provides a persistent Perl program to carry this out.
Listing 16.6 Template Naming and Notification (do_set_name.pperl)
01 use 5.6.0;
02 use strict;
03 use warnings;
04
05 use CGI ();
06 use Net::SMTP ();
07
08 my $q = CGI->new();
09
10 # get the template name
11 our $template = $q->param('template');
12
13 # remove the trailing ".psp" if present
14 $template =~ s/\.psp$//;
15
16 # check the assigned name
17 unless ($template)
18 {
19 error("Please specify a template name.");
20 };
21
22 if ($template =~ /[^A-Za-z0-9_]/)
23 {
24 error("The specified name contains illegal characters. "
25 ."Please specify a template name with letters, "
26 ."numerals or underscores only.");
27 }
28
29 my $filename = "$ENV{DOCUMENT_ROOT}/templates/$template.psp";
30
31 # write the temporary template file to the n
32 open(NEWFILE, ">$filename") or error("Can't write $filename: $!");
33 open(OLDFILE, "$ENV{DOCUMENT_ROOT}/templates/test.psp")
34 or error("Can't open temporary template file: $!");
35 print NEWFILE while (<OLDFILE>);
36 close OLDFILE;
37 close NEWFILE;
38
39 # send an email notification
40 my $smtp = Net::SMTP->new('mail.server.com');
41
42 $smtp->mail('user@company.com');
43 $smtp->to('user@company.com');
44
45 $smtp->data();
46 $smtp->datasend("To: user\@company.com\n");
47 $smtp->datasend("Subject: Template '$template' has been updated.\n");
48 $smtp->datasend("\n");
49 $smtp->datasend("The template named '$template' has been updated\n");
50 $smtp->datasend("and added as $filename.\n\n");
51 $smtp->dataend();
52
53 $smtp->quit;
54
55
56 # redirect the user to the upload page
57 print qq{Location: upload.psp\n\n};
58
59 undef $template;
60
61 sub error ($)
62 {
63 my $error = shift;
64 $error =~ s/\s/+/g;
65
66 # redirect the user to the naming page with an error
67 print qq{Location: set_name.psp?error=$error&template=$template\n\n};
68 exit;
69 }
Like Listing 13.4, Listing 13.6 is written in a persistent style, as declared in lines 01[nd]03. Lines 05 and 06 load the CGI.pm and Net::SMTP modules, respectively. Net::SMTP provides a pure-Perl interface to email servers. Line 08 creates the $q CGI object for form variable interaction. Line 11 then assigns the template form variable to the $template global variable declared using the our keyword.
A Note About our
Using the our keyword in a persistent context usually isn't advised because it overrides the protections offered by the strict pragma. In some cases, it's necessary to use global variables to simplify subroutine interaction, however. As long as the our declaration is combined with variable clean-up code (as used in Listing 13.6 in line 59), it is possible to use it in a persistent context without unexpected behavior.
After the $template variable is defined, it needs to be checked for a legitimate value to be used as the filename for the saved template. Line 14 removes any .psp extension added by the user to standardize the naming format. Line 17 makes sure the result actually contains a filename–on the odd chance that the user specified the name .psp or nothing at all. The template format then is checked by line 22 for any characters not allowed in the template filename. If illegal characters are found, the user is directed back to the entry form with an error message. Note at this point that the template name sent back to the entry form is the processed version (without any .psp extension) instead of the raw form variable. In most cases, it's better to provide the user with the original entered values if an error occurs, but in this case, the modified name is preferred because it indicates to the user one aspect of the proper format for further reference.
After the filename has been verified, lines 29[nd]37 write the new template to the file specified. It also would be possible to delete the temporary file using the unlink command after line 37, but in this case, it is left in place until the next uploaded template overwrites it.
After the file has been written correctly, lines 40[nd]53 send an email notification with the new template name and its saved location. This section of the program could just as easily log the action to a file, save the uploaded template to an archive, or perform any other action required after a template has been uploaded. After notification has been sent, the program redirects the user to the original upload page.
Template Modules on CPAN
In addition to the custom template managers made possible with embedded programming environments such as PSP and HTML::Mason, a number of template management modules are available on the Comprehensive Perl Archive Network (CPAN). These modules vary from simple variable interpolation modules to full-blown HTML document management systems with more variations introduced regularly. As this book was written, only a few of the template modules were available in released versions greater than 1.0.
One common aspect of the Perl templating modules available on CPAN is their Perl-centric design. This design contrasts with the Web-centric design of systems such as HTML::Mason and PSP. Many of the modules assume a usage that is similar to the CGI.pm module, with object methods to handle variable assignment and section definition. This approach doesn't mesh well with the embedded programming model, but it does enable templates to be used with persistent Perl code written in a CGI style.
Two of the common templating modules available on CPAN are HTML::Template and Text::Template. They take different approaches to the template language they implement–HTML::Template abstracts Perl completely away from the template language, whereas Text::Template uses Perl segments directly as the template language. In addition, Text::Template is designed for use with any text template, whereas HTML::Template is geared toward HTML specifically. However, the basic idea behind both is to provide a programmatic interface to filling templates with information stored in Perl data structures.
HTML::Template
The HTML::Template model is separated into two parts:
- Templates with a custom template language
- Program code that loads and manipulates the templates
The template language uses HTML-like tags to specify template sections and variables to interpolate. When templates are processed, the programming interface to HTML::Template provides a group of methods that act on the HTML::Template object created by loading the template into the Perl program.
Templates in this system consist of any valid HTML file with extra HTML::Template tags specifying sections to interpolate programmatically. The most basic tag used to interpolate values is <TMPL_VAR>, which specifies a variable to be replaced with the value specified in a Perl program that loads the template. For instance, the following code section would replace the variable fullname with a name provided by the Perl program:
Listing 16.
<p>The officer in charge was <TMPL_VAR NAME="fullname">.</p>
With the appropriate code in the persistent application, the <TMPL_VAR> tag would be replaced by a full name. The code for this uses the param method of the template object created by loading the template into HTML::Template. To replace the <TMPL_VAR> tag with the full name value and display the result, for instance, the code might look like the following:
Listing 16.
my $template = HTML::Template->new(filename => 'officer.tmpl');
$template->param( fullname => $full_name );
print $template->output;
The main difficulty with HTML::Template arises from its oversimplified template language and CGI-style module interface. The module interface is too complex to provide a simple templating system to HTML-style documents and the template language is too simple to be a dynamic template manager. An indicator of this tension comes from the other tags used by the templating language: <TMPL_LOOP>, <TMPL_IF>, and <TMPL_ELSE>. These tags are included to provide Perl-like constructs for complex templates, but they work only somewhat like their Perl counterparts. To use the template constructs the way the analogous structures in a Perl program would be used, the Perl code in the main application has to become more complex and, potentially, more difficult to maintain. Similarly, HTML::Template provides no facility for creating tags other than the core set provided, so Perl-generated sections of the template can't be included without first assigning their output to a variable and interpolating the variable through the param method.
Text::Template
Another approach to template processing in Perl is exemplified by the Text::Template module. This module is designed for use with any sort of text document, HTML or otherwise. The module could just as easily be used to fill names into an automated email system or any other generated text document. To assist this process, the module provides an interface to load templates from most sources of text, including files, file handles, and Perl variables.
Text::Template is similar to embedded Perl solutions on the surface. Rather than use a special syntax for the template language, Text::Template uses Perl sections enclosed in curly braces. When the template is processed, the Perl sections are evaluated and replaced with their values. The current values of variables in the calling program are used to interpolate any variables listed in the template, and other Perl code in the template is executed as listed. For instance, this template segment would replace the specified Perl section with the contents of the $full_name variable:
Listing 16.
<p>The officer in charge was {$fullname}.</p>
However, Text::Template is implemented in a manner that is different from embedded Perl environments. Rather than being invoked directly and interpolated inherently by the invoking environment, a Text::Template template is loaded explicitly by Perl code that resembles CGI style more than an embedded Perl environment. For instance, replacing the $fullname variable with its value would require a code segment like the following:
Listing 16.
my $template = Text::Template->new(TYPE => FILE, SOURCE => 'officer.tmpl');
my $full_name = 'Officer Krupke';
my $text = $template->fill_in();
print $text;
This subtle difference between embedding systems and Text::Template means that templates written for Text:Template potentially could be used by CGI-style programs as well as embedded Perl programs. In addition, templates could be created for any number of page sections and then evaluated one at a time by the Web application to create a complete HTML page. This makes Text::Template a good basic toolkit for creating customized template systems, if not a complete template manager.
The major advantage to using Text::Template also is its biggest drawback. Using Perl as the template language provides a robust language that's fully compatible with the underlying Web application code. However, the main point of abstracting code out of a template is to provide template access to site designers who don't know–or wish to learn–Perl. In addition, the Perl segments in the template have no additional structure or syntax added to enable abstraction of Perl constructs beyond simple variables. This is likely to lead to more Perl in the template than with which site designers are comfortable. It also leads to less abstraction, which is contrary to the goal of templates.
Other Template Modules
CPAN is full of other Perl modules that provide the same type templating features in various implementations. Modules such as CGI:FastTemplate, HTML::FormTemplate, HTML::DynamicTemplate, the Template Toolkit, and others all provide some subset or superset of the functions required to create a templating system, and many more are likely on the way. This profusion of template interfaces probably is due to the highly custom nature of template systems–no one templating approach is good for every circumstance, so finding the right approach to templating always requires some searching and evaluation.
Performance Issues with Templates
As can be expected with any system that saves programming time and does real work, most template systems involve additional processing overhead. The types and amount of overhead vary depending on the template manager and Perl environments used, but most template systems use a bit more CPU time to process and interpolate a template and a little more memory to store the data structures used to store templates. Some template managers can add considerable overhead if templates are being interpolated inefficiently, however. Testing the performance of a few candidate template systems against a suitably complex template and Web application is generally recommended.
Caching templates is the most important performance improvement a template manager can provide. Performance can suffer significantly if templates are loaded off disk with each request, just as CGI programs lose performance if they are loaded and processed with each request. Some template managers cache templates automatically, but others are not designed for a persistent environment. In the latter case, it's sometimes necessary to implement a caching mechanism that stores processed template objects and reuses them with each request.
Scheduled Publishing with LWP
Although templates are useful even on a purely static site, the overhead of template processing might be high relative to simply serving static HTML pages. For high traffic sites where overhead due to template processing is prohibitive, a good alternative is scheduled publishing.
Not all pages need to be generated dynamically for each request. Following the rule of caching information whenever possible, it might be a good idea to precache dynamic pages as HTML instead. Depending on the frequency of updates to the underlying information, pages can be published on a schedule that can reduce processing overhead by a few orders of magnitude. For instance, a news site that gets a hundred home page requests per second might update its headline database only every five minutes. Instead of accessing the database to generate a home page with headlines for each request, the site can publish the page to HTML once a minute. The resulting overhead would drop by a factor of 6000.
Publishing dynamic pages to HTML from a Perl program is made possible by the LWP module. LWP provides an interface to Web client operations, including retrieving a Web page over the network. A simple program can be written to access the dynamic Web page as a browser and save the result to disk as a static HTML file. This kind of program then can be scheduled to run at regular intervals to provide a relatively current snapshot of the page being generated.
Incidentally, most embedded Perl environments provide a method of caching the result of dynamic pages on an individual basis. This enables the pages to be accessed with a dynamic naming scheme instead of publishing some pages to HTML in new locations and keeping others in their original locations. These cached pages usually incur minor overhead because the Perl environment can't cache the pages as efficiently as the Web server can, but the resulting flexibility usually is worth this minor performance difference.
Sidebar: Templates for the Lazy
Before I became aware of these fancy templating methods–in fact, before any of them were developed–I was faced with the task of implementing something that would offer all the benefits of a template without the work involved in creating a real template system.
My first solution was a distressingly simple one, implemented in the heady Web days of 1996 when dynamic sites were new and I had too much free time at work. The acute problem at that time was how to write as little HTML by hand as possible while still creating a commercial-quality site. Most sites at the time were composed entirely by hand, but that meant lots of tedious cutting and pasting whenever a menu or a background color was changed. I was keen on simplifying this process down to a few clicks and a Perl script. Compounding the problem was the need to do some of my work remotely from a terminal that had only basic Web access.
The solution I developed was dubbed Wackel, and it was a simple solution indeed. A repository of HTML fragments was kept that could be edited individually from an HTML form. These fragments–which consisted of a header file, a footer file, and the core segment from each existing Web page–could be compiled in real time to test the resulting HTML for completeness. This virtual site also could be used as a staging area for new designs and page modifications. After modifications to the staging site were approved, it then was possible to publish the compiled HTML files to the live site in their completed form. Note that none of the files was actually dynamic in nature–the whole system was designed simply to administer a static site.
Later, I discovered VelociGen (the software, not the company) and a little gem of a function called vepload, which did the work of Wackel in real time and with half the lines of code. Pages could be broken into their header, core, and footer sections, and the header and footer could be called from each core page directly through the vepload function embedded in a <perl> block. This also opened up the possibilities of embedded Perl code in HTML, and I was off and running. This technique still is a valid way to implement templates, even if the resulting template files aren't quite valid HTML, and I still use a variation of it to create the pages on my own site. (Of course, I'm bound to start using the newer templating methods Real Soon Now.)
Summary
Templates are a natural extension of embedded program that enable HTML formatting to be abstracted away from Perl code. The resulting interface gives both site designers and Web application developers the freedom to design their respective parts of Web applications without requiring input from the other group. Templates can be implemented in a number of ways, but the common approach is to create an HTML-like syntax for the template document, which then is interpolated by code residing in the Web application or a separate template management system. Simple systems can involve a single template document for all Web applications or static files, while more complex template systems use many dynamic templates that customize their output to suit requests. A simple template system can be written trivially in embedded environments such as PSP and HTML::Mason, or any number of existing template modules can be downloaded from CPAN for use with embedded or CGI-style programs. Templates likely are to incur a minor performance penalty, but the resulting flexibility in site design usually is more than worth the overhead.