Built with Sitecore » Development

Using Lucene to Query Sitecore’s Master Database

Dave P. — Mon, 18 Feb 2013 15:31:40 +0000

Sitecore Searcher via The Sitecore Searcher and Advanced Database Crawler framework, extending the Sitecore.Search API, allows for direct queries against Lucene indexes. Running queries against Lucene becomes simple given the Sitecore Searcher framework. Notice the signature of the GetItems method below:

public virtual List GetItems(IEnumerable parameters, bool showAllVersions = false, string sortField = "", bool reverse = true, int start = 0, int end = 0)

Executing GetItems given the default values for the optional parameters will return records matching the query provided. Everything works as expected, when querying against items in the Web database. Running the same code against items within the Master database as the indexing source, if not careful, can produce unexpected results.

The default settings for querying Lucene will return the first version of every item. While this holds true for the Web database, where every item has only one version, the Master database, however, will be indexed containing more than one version. Querying the Master database using the default settings will return version one for every item.

Resolving this issue involves adding an additional search parameter to include only the latest version of every item. The Sitecore API includes a built in field to allow this. See the FieldName property setter below:

private FieldSearchParam GetLatestVersionSearchParameter()
        {
                 var isLatestVersionParameter = new FieldSearchParam() {                  
                         FieldName = Sitecore.Search.BuiltinFields.LatestVersion,
                         FieldValue = "1",
                         Condition = QueryOccurance.Must
                 };

                 return isLatestVersionParameter;
        }

Appending the parameter above to the list of ISearchParam objects will tell the Sitecore Search API to include only the latest version of every item.

Sitecore-ISITE Design Webinar 4/24: Avoid Site Killers with Effective CMS Planning

daponovich — Wed, 11 Apr 2012 05:01:03 +0000

If there’s a fundamental fact organizations ignore at their peril when creating a digital strategy and implementing a web content management system, it’s this: This stuff is difficult to get right. It’s not impossible, but there are enough landmines in any project that you need to plan rigorously for success.

That’s the spirit behind an upcoming ISITE Design-Sitecore webinar intended to help you prepare wisely for your next web content management initiative and avoid so-called “site killers” that can make your projects go off the rails.

The webinar on April 24, “Avoid Site Killers with Effective CMS Planning,” will benefit anyone implementing a new CMS in 2012 or rethinking a current CMS. The event complements a recent Sitecore White Paper in which ISITE Design’s Jeff Cram addresses the important (often overlooked) role of web governance planning.

Jeff, ISITE’s co-founder and publisher of ISITE’s sibling blog, The CMS Myth, will highlight five site killers that threaten to sink your CMS project. You’ll learn how effective web governance planning can make your organization happier and more productive with CMS.

The webinar will cover make-it-or-break-it governance issues:

Making a real plan and asking, “How will we use the CMS?”
Planning for large-scale roll-outs of dozens (or hundreds) of sites
Aligning customer experience planning for multi-channel success
Finding common ground between Marketing and IT
Preparing for “day two” and post-launch success

When: April 24, 2-3 pm Eastern (11-12 Pacific)

Where: Register here for the webinar

Download the related white paper: “Content Management: The (New) Governance Manifesto”

Getting Workflow Working

Rick Cabral — Mon, 09 May 2011 18:01:02 +0000

As a solutions architect, I have had the opportunity to review a diverse set of Sitecore installations authored by folks all over the United States. One common source of implementer and administrator consternation involves successfully implementing Sitecore’s Workflow feature. Unfortunately, the name “workflow” is charged with dozens of subtle expectations. Implementation is further complicated by the fact that Sitecore does not explicitly train developers on the use of the Workflow feature. It’s an exercise “left to the reader”. These are unfortunate circumstances; at its heart Sitecore’s Workflow system is very simple, and requires minimal setup for success. In this post, I’m going to cover the rules of thumb to get a sustainable Workflow implementation running on Sitecore.

Know the correct use for Sitecore Workflow

At its heart, the Workflow feature’s sole purpose is to flag a given version of a content item as being finalized, ergo: approved for publication. Workflow design should be anchored around this fact. Workflow should have one (and only one) “Workflow State” marked “Final”. This “final” workflow state should be the last workflow state for a given workflow. Many developers try to “disable” workflow by marking all states as final, or introduce a “final” workflow state at the beginning of workflow for a “publish-then-approve” effect. Both implementations have consequences that confuse the user and clog up the content database.

Aside from the obvious reasons for ending Workflow in a single, final state, there’s an architectural assumption in Sitecore that cannot be ignored. The “final” workflow state signals Sitecore that an Item version is no longer available for editing. If a user attempts to edit an Item Version that is in a workflow state marked final, Sitecore will create a new Version of that Item for the user. In the scenarios listed above, the original versions will never progress to a “finalized” state from the user’s perspective, and will remain to clutter the workbox indefinitely.

Recommendation: One workflow state marked final at the end of the workflow process, to indicate that an Item version has progressed through review and is approved for publication.

Know how to assign Workflow to a Template’s Standard Values

Workflow should always be assigned to a Standard Values Item using the Review tab, as shown below.

Avoid accessing the Standard Fields to set Workflow, as it’s a common source of developer confusion. If Workflow is set using the Ribbon, this is what the Standard Fields should look like:

Note that the Workflow and State fields are blank. This is correct for a Standard Values item, as these fields reflect the Workflow and State for a particular instance of an Item type. If you’re looking for a hint, look for Standard Values items in the Workbox, where they normally would not appear. Chances are, someone filled the Workflow and State fields accidentally. Clear them to return your installation to normal.

Shunt between Workflow States, not Workflows

In the image above, note that the Workflow field is “shared”. This indicates that all Versions in all Languages of a given Item will use the value of this field. If you programmatically alter the value of this field, the effect is universal. This is a good hint that architecturally, Sitecore was not designed to support moving Items between workflows. The solution? Consolidate all the various workflow states across all disparate workflows and use Workflow Commands to select the appropriate path between Workflow States. In addition to being more Sitecore-compliant, this solution has the benefit of being code-free.

Separate Security Concerns to Reduce your Workflow Complexity

While Sitecore Security is a post all its own, it merits consideration here. Sitecore uses three different security concerns to evaluate the authority of a user’s actions:

When designing your security scheme, ensure that each of your security roles addresses one and only one of the concerns above. One should note that Sitecore does not follow this advice itself. The “Sitecore Client Authoring” role is shipped with a set of Workflow security settings that need to be cleared in order to establish the separation described above.

In theory, all users belong to three security roles, one for each of the concerns above. The first role specifies access to Sitecore features, the second the rights for Items in the content tree, and the third access to various Workflow states and Actions.

Benefits of this security model:

Allows you to add access to Sitecore features without having to worry about a user’s Item security
Allows you to create fine-grained access to parts of the Content Tree without worrying about workflow permissions
Allows you to promote a user’s workflow rights without changing any other aspect of their security scheme.
In most cases, allows you to use a single workflow for the entire installation.

The last point is key. In order for a User to see an Item in the Workbox, they need “write” access to the Item as well as the language(s) of relevant versions. The user also needs “Workflow Write” access to the Workflow State of the relevant version(s). To move those versions to a new Workflow State, they need Workflow Command Execute rights on at least one Command for that Workflow State. Understanding this architectural detail allows workflow to be extremely flexible without creating multiple workflows. One simply creates security roles for a number of workflow personas (tyipcally this involves an editor and an approver role). These workflow roles apply permissions only to objects in Sitecore/System/Workflow and therefore can be used by any person who has access to any part of the content tree. It is the content tree security that refines what the user “sees” in the Workbox.

Takeaways

We’ve covered a lot of ground on Workflow philosophy. Here’s the digest version:

Use workflow to control whether a Version of an Item is publishable
Set the default workflow in the Standard Values of your Data Templates using the Ribbon, not the Standard Fields
Create a single Workflow for your project and use Workflow States to manage content lifecycles
Separate Workflow security from Item security in your Role definitions

Follow these simple steps to simplify your life when it comes to Sitecore administration. Happy configuring!

Crowning the Sitecore MVPs

Jeff — Tue, 19 Apr 2011 20:23:08 +0000

There’s up to 20 Sitecore MVPs mingling among the crowd at Dreamcore. Our own Rick Cabral is one of them, although I haven’t even seen him yet today because we stay on different sides of the track (biz vs tech).

Here’s a suggestion: Let’s give em all Burger King crowns so we can easily ID them. Someone want to run out and get 20? I’m sure the MVPs would be thrilled to wear em, right Rick?

Tweak your Sitecore Delivery Environment

Rick Cabral — Mon, 18 Apr 2011 15:37:35 +0000

Your Sitecore project is ready to roll, the presentation layer is done, and the code looks good:

You’re using Fast Query and New Search to retrieve lists of Items
Your layouts are not overly stacked
All Renderings and Sublayouts have appropriate output cache settings

There are, however, a few things you can still do to really make your installation perform. After all, no one can predict when your client will be front-page news, or the target of the newest digital meme. Take advantage of this one-stop post to turn your Sitecore install into a screen-rattling dragster of a web site.

Use Windows Server 2008 64-bit Edition

There is a noticeable improvement running Windows (and .NET) on 64-bit systems. All modern CPUs support it, make sure you’re getting the value out of that server, virtual or otherwise.

Enable Static and Dynamic HTTP Compression on your IIS Server

Next to running 64-bit software, HTTP Compression is the easiest and fastest way to impact perceived application performance. For this post, I’m going to assume you took my advice on the first topic, and limit the implementation to IIS 7.x, which ships with HTTP Compression! The IIS 7 control panel features options for enabling compression of static and dynamic content. You want to turn both of those on. There’s a catch! Dynamic compression is not installed by default on Windows 2008. Head for Settings and use Programs and Features to turn on this specific feature of IIS.

Strip Unnecessary HttpModules and HttpHandlers from your web.config File

Unless you are integrating with certain security systems, the following modules can generally be turned off on a Sitecore installation:

WindowsAuthentication
PassportAuthentication
AnonymousIdentification
UrlAuthorization
FileAuthorization

Simply remove them from the element in your web.config. Note that the last two have implications; your *.config files may be vulnerable unless you have a well crafted extension handler stack. Consider yourself warned.

Tweak Threads in the machine.config File

By default, .NET web applications are throttled back to allow for multiple applications and services to run on the server. Because I’m assuming we’re configuring a dedicated Sitecore web server you can boost the amount of memory, worker threads, and IO threads available to ASP.NET. In your machine.config find system.web/processModel and remove all attributes, then add the following:

The above settings assume a 4 core CPU (real or virtual) and at least 4GB of RAM.

Ensure Sitecore Always has Sufficient HttpConnections

Creating a connection costs more than using an existing one from the pool. Setting these properties will ensure higher performance for required resources

In your machine.config find system.net/connectionManagement and remove the default settings. Add the following:

Maxmind is the geolocation service used by Sitecore’s OMS tool by default. Because the service is called for every request (if you’re using OMS) you want that service to be as available as possible.

The numbers above come from Microsoft’s “secret sauce” recipe of 12 connections per CPU.

Ensure Sitecore Always has Sufficent Resources to Handle Requests

In the web.config file, find the httpRuntime element and replace the current attributes with the following:

Note that “executionTimeout” is set very aggressively. Test your application thoroughly to ensure all operations complete. The “maxRequestLength” setting will help protect you from certain types of HTTP attacks, but will also prevent your content authors from uploading sizable media library items. This setting should only be used on content delivery servers.

Set Cache Sizes

Sitecore’s as-shipped cache configuration is… conservative. Add lots of RAM to your server and crank it to eleven! In the web.config adjust the default cache settings:

Then adjust the output cache settings for each site in your installation. There is a dedicated element:

Or you can set output cache size attributes on the site node, discussed next.

Tune the Settings for Each Site in your Installation

There are a plethora of settings here. Check out this example, which is optimized for a content delivery server:

Footnotes

A certain amount of research was required to develop these settings, your result may vary. Consider the following to be required reading:

This post was heavily inspired by a similar post on JustAgile.com that takes more of a sightseeing approach.

Anatomy of a Successful Sitecore User Group

daponovich — Sun, 17 Apr 2011 16:16:42 +0000

The annual Sitecore Dreamcore conference hits Boston this week, April 19-20, with a sold-out venue and an international contingent of speakers and attendees. Although we don’t have hard data to back it up, the Boston region likely has the highest concentration of Sitecore developers using the platform anywhere this side of Denmark.

The clearest sign: as of this writing, the New England Sitecore User Group boasts 139 members – believed to be the largest Sitecore user group in the world. And it’s an active bunch, reflecting a multitude of Sitecore customers and partners work and live in the area.

ISITE Design launched the UG in Fall 2009 and hosts a monthly confab at our Cambridge office, just across the river from Boston. ISITE’s Sitecore MVP Rick Cabral leads the pack (typically 25-30 attendees each month) addressing best practices around topics like multisite Sitecore deployments, workflow and security, fine-tuning your CMS performance and more.

But contributions and example projects from the community show the range of projects and depth of involvement in the platform and put the “users” in user group. Everyone comes away a little smarter, a little more energized.

We’ve been asked by folks in other cities what makes for a successful user group. There’s no magic bullet – we’re fortunate to have a robust base of users around Boston – but there are some lessons to offer.

Make it consistent. Meet regularly at a central location if possible. Being near public transport helps a ton. We typically meet monthly, on a mid-month Wednesday night. Pizza at 6; presentation at 6:30.
Make it compelling. Define topics that the audience wants/needs to talk about. Create real, focused content highlighting real world examples that people can learn from. Rick drives a core topic every month, but the floor is always open to others to bring examples and talk about challenges and ideas they’re facing.
Make it fun. Main presentations usually last no more than 75 minutes, with plenty of time to network before and after. People talk shop, discuss job opportunities, recent projects. Then the group repairs to a local watering hole for more socializing.
Make it about the community. Create an environment where many people can contribute, minus a sense of competition. It’s true: many attendees from among the partner ranks compete against each other for client projects. But a well-run user group encourages camaraderie that transcends competition and creates a collegial atmosphere for continual learning.

If you’re heading to Boston for Dreamcore, join the fun. On Monday April 18, 7-10 pm, the New England User Group (and ISITE Design) hosts a Dreamcore Meetup at Minibar in Copley Square. You can register here.

Five Years Working with Sitecore

Rick Cabral — Thu, 14 Apr 2011 14:08:47 +0000

I looked at Sitecore as a CMS product back in October of 2005. My team trained that December on Sitecore 5.1, and after the new year, we were implementing the (then brand new) 5.2 version. Five years and many installs later, it’s hard to even conjure up the old 5.2 interface. I remember it being dark, full of very small input fields, and lots of sliders. Interestingly enough, most of the core features have survived: the “Desktop”, Data Templates, Field Sections, Field Types, the locations of things in the Content Tree, and Workflow are all recognizable (if a little less feature-rich).

Evolution

When asked to consider what’s changed, you’d need to insert some qualifiers. Architectural changes have an impact on the core product, while finesse changes impact the system users (developers and content authors alike).

Sitecore as a platform has been remarkably consistent. Off the top of my head I can think of only two major architecture changes: the re-casting of Masters to “Branch Templates” and the introduction of “Standard Values”. Both of these were life-changing for the developer and really improved the data design aspect of the product.

The list of finesse changes is long: Office 2007-style interfaces, the data designer, Page Edit Mode, and a host of behind-the-scenes database and API improvements have radically changed the implementer’s workload. For the practiced developer, ramping up a new install and prototyping a site on Sitecore has become a demonstration of rapid application development. For the newly trained, Sitecore’s documentation provides a wealth of best practice information and deep-dive recommendations on the most complex aspects of the system.

Personal Expectations for the Future

Working in content management for five years, I’m obviously something of a Sitecore fan boy, and was recently recognized as such. That said, there are some improvements I’d like to see in the product:

Redesigned User Experience for Tabular Data

At its heart, Sitecore’s data is modelled after a hierarchical concept similar to XMLDom and OOP. I credit this architecture as the reason programmers tend to fall in love with the system immediately. It’s absolutely fabulous for creating page templates with generalizations, reusable components and marketing-friendly site maps. However, the tree-like structure has some limitations when representing content that marketers tend to think of in a “siloed” or tabular fashion; optional content, comments, and document archives come to mind. While this kind of content can be modeled in Sitecore, the developed model exposes too much of the content storage and url-management overhead to users. Interface customizations are possible, but expensive to develop.

Task-Based Authoring Interfaces

When you have some gnarly business requirements, Sitecore’s flexibility is a blessing. The API lets you tap into any housekeeping pipeline you can imagine. It’s almost a shame that their control stack is (at time of writing) a relatively closed option. the Sheer API is not documented, and although remarkably clever (when decompiled), it is very hard to create a data entry customization that is forward compatible. Sitecore’s answer to date has been Page Edit Mode, which gives content authors that “what you see” editing experience. In reality the demonstrable aspects of Page Edit Mode are either low hanging fruit (click-to-type) or beyond all but the most web-savvy marketer (custom rendering rules). In the future, I see developers needing a way to “step off” from Page Edit Mode and launch their own dialogs & wizards to request information in a way that is more in line with business objectives.

Wrapping it Up

Interestingly enough, the above desired improvements are probably “on the roadmap” for future releases of Sitecore. For those in mid-implementation, it would also be reasonable to have developers go “off the reservation” and develop their own custom data entry interfaces. The API supports the concept easily. Just be sure to estimate the effort accurately.

As a seasoned Sitecore developer, I remain impressed with Sitecore’s base architecture. The flexibility and power of the product makes it a joy to work with. The developer ecosystem is growing rapidly, and most developers working on the platform are passionate enthusiasts. Increasingly I see Sitecore itself leveraging its own product for point solutions like OMS and Email Marketing. The fact that these solutions can be implemented seamlessly on the core product using nothing more than the existing interfaces, data storage mechanisms, and APIs is certainly a tribute to a well designed system. It’ll be interesting to see what they’re keeping behind the curtain at Dreamcore. Hope to see you there.

Dreamcore 2011 Approaches

Rick Cabral — Mon, 11 Apr 2011 19:27:02 +0000

If you’re into Sitecore and you’re a developer like me, Sitecore’s annual Dreamcore event offers a great opportunity to hear the minds behind the product speak about the technology, and the product roadmap. This year’s schedule does a nice job of keeping the developer talk track free of distractions. I plan on camping out there, and so should any certified Sitecore developer.

Lars Nielsen, John West, Kerry Bellerose, Alex Shyba and Alexey Rusakov are all giving talks on current or future technology at a level Sitecore geeks will appreciate. If you’ve never met or watched John West demonstrate, it’s a must-attend.

Been working with Sitecore for a few years and have some opinions about the product? Dreamcore is repeating the Technology Interact Panel, which gives implementers a forum for direct Q&A with Sitecore leadership. Last year’s panel had some energy in it, bring your opinions and sit near the front of the room.

Of course, the presentations are only half the fun. Whether you are a lone wolf developer or a member of a team, it’s quite an experience to hang out with a few hundred other people that know exactly what you do for a living. Come pick brains and exchange business cards. Better yet, come join the local Sitecore User Group for our kickoff party. Hope to see you there.

Sitecore Multisite Right

Rick Cabral — Fri, 01 Oct 2010 01:41:13 +0000

Wednesday at the monthly Sitecore User Group (New England) We covered an archetypal multi-site solution, the ISITE way. For those who couldn’t attend, I wanted to take a few minutes and cover the highlights.

Multisite Concerns:

Site-based security roles
Site-based Layouts/Sublayouts/Renderings
Site-based Data Templates
Site-based Insert Options (and therefore IA)
Site-based digital assets
Site-based code libraries
Multiple simultaneous development projects
Distributed developers

ISITE’s approach to the above concerns is holistic. If you’re going to run a Sitecore install with 50+ web sites of various sizes and design, you need a plan. We’ve been there, we’ve done that. Here’s what the scars have taught us:

Decide what all your sites must have in common.

Let’s face it, Sitecore is a single ASP.NET application running on your web server. You have to plan carefully to prevent namespace collisions, duplicate functionality, and general code rot before your content authors ever get a look at the system.

Take the time to imagine a basic “spine” for your web site, as design agnostic as possible. What navigation paradigms will you need to support? What kinds of content are common across all sites? News? Events? Blogs? Profiles? Take a least-common-denominator approach. If possible, build a “white label” web site utilizing these components. The data and presentation defined here will serve as your base library.

Leverage Sitecore’s OOP Goodness

Treat Data Templates as if they were .NET Interfaces. Wrap Items in some strongly typed business objects and leverage polymorphism to give your baseline functionality opportunities to stretch to fill new needs. If one of your sites has something that isn’t in your core framework, stop and consider whether it’s a one-off, or whether it should be folded into your core feature set.

Enforce Best Practices with Abstract Controls

How do you ensure that the output of a control is cached? How do you ensure that programming intern next to you has turned off Viewstate for all the controls she’s building? You’ve standardized the HTML output for a blog roll, how do you make sure it’s used on the next site with that radically different color scheme?

You build a control library of course! You build it so that programmers are forced to implement your best practices with every derivative. Yes, it takes time to train developers where to look in your library, but it sure saves time on the QA bench. If your team is using Continuous Integration and code grooming tools like FXCop, you can even automate the enforcement of your conventions.

Don’t Forget the Authors

Within a multisite installation, nothing is quite so tedious as adding an image to a page when you have to sort through the folders for every site on the installation. Use Sitecore Query to set the Source property of your Image, File, Treelist, Multilist, and Droplink controls to reduce the scope of the result set. Your wrists will thank you for it.

A Nod to Open Source

Ruby is many things to many people. I’m not going to share my opinion, but I do endorse one of its core design philosophies: Convention over Configuration. Name objects that behave similarly the same between sites. Keep your site folder structures consistent in Sitecore and in the file system. I’m not going to give away the farm here, but with convention on your side, you could use context and a little pipeline magic to significantly cut down on the number of “administrative” objects you have to create in Sitecore.

Get Agile, Get Pragmatic!

Requisitioning a site in Sitecore takes a lot of brute-force data manipulation:

web.config settings
Data Template derivation
Standard Value overrides
Layouts on a site-by-site basis
Security Roles & Security settings
File system changes
Visual Studio Solution changes

For one of our clients, it’s more than 50 steps to get a new site running. Automate it. Learn to use the API to create templates, layouts, roles, etc and make your life a lot easier. Don’t forget to provide a means to reverse your changes.

That’s a Wrap

We’ve covered a number of ISITE’s core philosophies for successsfully implementing multisite, but this was most definitely a survey course. For further details, hit me up at the next User Group in Boston. Join our group today and come hang with some serious Sitecore geeks.

When will Azure support be available?

Rick Cabral — Mon, 19 Jul 2010 17:36:59 +0000

Apparently not until the end of the quarter. Check out this article from Fierce, which summarizes a number of public announcements from Sitecore last week.

At Dreamcore I seem to remember Lars mentioning that there were “several” point releases in the queue before Massive arrives…