drupal

Fixing Drupal Fast - Using Ansible to deploy a security update on many sites

Earlier today, the Drupal Security Team announced SA-CORE-2014-005 - Drupal core - SQL injection, a 'Highly Critical' bug in Drupal 7 core that could result in SQL injection, leading to a whole host of other problems.

While not a regular occurrence, this kind of vulnerability is disclosed from time to time—if not in Drupal core, in some popular contributed module, or in some package you have running on your Internet-connected servers. What's the best way to update your entire infrastructure (all your sites and servers) against a vulnerability like this, and fast? High profile sites could be quickly targeted by criminals, and need to be able to deploy a fix ASAP... and though lower-profile sites may not be immediately targeted, you can bet there will eventually be a malicious bot scanning for vulnerable sites, so these sites need to still apply the fix in a timely manner.

In this blog post, I'll show how I patched all of Midwestern Mac's Drupal 7 sites in less than 5 minutes.

St. Louis Drupal Group - Hackathon on Headless Drupal 8 & AngularJS

Now that Drupal 8.0.0-beta1 is out, and the headless Drupal craze is in full-swing, the Drupal St. Louis meetup this month will focus on using Drupal 8 with AngularJS to build a demo pizza ordering app. (The meetup is on Thurs. Oct. 23, starting at 6:30 p.m.; see even more info in this Zero to Drupal post).

We'll be hacking away and seeing how far we can get, and hopefully we'll be able to leave with at least an MVP-quality product! I'll be at the event, mostly helping people get a Drupal 8 development environment up and running. For some, this alone will hopefully be a huge help, and maybe motivation to adopt Drupal 8 more quickly!

If you're in or around the St. Louis area, consider joining us; especially if you would like to learn something about either Drupal 8 or AngularJS!

P.S. To those who have been emailing: the rest of the Apache Solr search series is coming, it's just been postponed while I've started a new job at Acquia, and had a new baby!

Solr for Drupal Developers, Part 2: Solr and Drupal, A History

Drupal has included basic site search functionality since its first public release. Search administration was added in Drupal 2.0.0 in 2001, and search quality, relevance, and customization was improved dramatically throughout the Drupal 4.x series, especially in Drupal 4.7.0. Drupal's built-in search provides decent database-backed search, but offers a minimal set of features, and slows down dramatically as the size of a Drupal site grows beyond thousands of nodes.

In the mid-2000s, when most custom search solutions were relatively niche products, and the Google Search Appliance dominated the field of large-scale custom search, Yonik Seeley started working on Solr for CNet Networks. Solr was designed to work with Lucene, and offered fast indexing, extremely fast search, and as time went on, other helpful features like distributed search and geospatial search. Once the project was open-sourced and released under the Apache Software Foundation's umbrella in 2006, the search engine became one of the most popular engines for customized and more performant site search.

As an aside, I am writing this series of blog posts from the perspective of a Drupal developer who has worked with large-scale, highly customized Solr search for Mercy (example), and with a variety of small-to-medium sites who are using Hosted Apache Solr, a service I've been running as part of Midwestern Mac since early 2011.

Timeline of Apache Solr and Drupal Solr Integration

If you can't view the timeline, please click through and read this article on Midwestern Mac's website directly.

A brief history of Apache Solr Search and Search API Solr

Only two years after Apache Solr was released, the first module that integrated Solr with Drupal, Apache Solr Search, was created. Originally, the module was written for Drupal 5.x, but it has been actively maintained for many years and was ported to Drupal 6 and 7, with some relatively major rewrites and modifications to keep the module up to date, easy to use, and integrated with all of Apache Solr's new features over time. As Solr gained popularity, many Drupal sites started switching from using core search or the Views module to using Apache Solr.

Solr for Drupal Developers, Part 1: Intro to Apache Solr

It's common knowledge in the Drupal community that Apache Solr (and other text-optimized search engines like Elasticsearch) blow database-backed search out of the water in terms of speed, relevance, and functionality. But most developers don't really know why, or just how much an engine like Solr can help them.

I'm going to be writing a series of blog posts on Apache Solr and Drupal, and while some parts of the series will be very Drupal-centric, I hope I'll be able to illuminate why Solr itself (and other search engines like it) are so effective, and why you should be using them instead of simple database-backed search (like Drupal core's Search module uses by default), even for small sites where search isn't a primary feature.

As an aside, I am writing this series of blog posts from the perspective of a Drupal developer who has worked with large-scale, highly customized Solr search for Mercy (example), and with a variety of small-to-medium sites who are using Hosted Apache Solr, a service I've been running as part of Midwestern Mac since early 2011.

Why not Database?

Apache Solr's wiki leads off it's Why Use Solr page with the following:

If your use case requires a person to type words into a search box, you want a text search engine like Solr.

At a basic level, databases are optimized for storing and retrieiving bits of data, usually either a record at a time, or in batches. And relational databases like MySQL, MariaDB, PostgreSQL, and SQLite are set up in such a way that data is stored in various tables and fields, rather than in one large bucket per record.

In Drupal, a typical node entity will have a title in the node table, a body in the field_data_body table, maybe an image with a description in another table, an author whose name is in the users table, etc. Usually, you want to allow users of your site to enter a keyword in a search box and search through all the data stored across all those fields.

Drupal's Search module avoids making ugly and slow search queries by building an index of all the search terms on the site, and storing that index inside a separate database table, which is then used to map keywords to entities that match those keywords. Drupal's venerable Views module will even enable you to bypass the search indexing and search directly in multiple tables for a certain keyword. So what's the downside?

Mainly, performance. Databases are built to be efficient query engines—provide a specific set of parameters, and the database returns a specific set of data. Most databases are not optimized for arbitrary string-based search. Queries where you use LIKE '%keyword%' are not that well optimized, and will be slow—especially if the query is being used across multiple JOINed tables! And even if you use the Search module or some other method of pre-indexing all the keyword data, relational databases will still be less efficient (and require much more work on a developer's part) for arbitrary text searches.

If you're simply building lists of data based on very specific parameters (especially where the conditions for your query all utilize speedy indexes in the database), a relational database like MySQL will be highly effective. But usually, for search, you don't just have a couple options and maybe a custom sort—you have a keyword field (primarily), and end users have high expectations that they'll find what they're looking for by simply entering a few keywords and clicking 'Search'.

Moving on to Acquia

I wanted to post this here, since this is more of my sounding board for the Drupal community, but the details are on my personal blog: starting October 6, I will be working for Acquia as a Technical Architect in their Professional Services group!

What does this mean for this site/blog, Hosted Apache Solr, and Server Check.in? Not much, actually—they will continue on, likely at the same pace of development they've been for the past year or so (I'll work on them when I get an odd hour or two...). I am still working on completing Ansible for DevOps, and will actually be accelerating my writing schedule prior to starting the new job, since I'll have a little wedge of free time (a.k.a. unemployment!) between Mercy (my current full-time employer) and Acquia.

I'm excited to start working for Acquia, and am also excited to be able to continue doing what I love—working on Drupal, working on Solr, working on Ansible/infrastructure, and working in their respective communities.

Multi-value spatial search with Solr 4.x and Drupal 7

For some time, Solr 3.x and Drupal 7 have been able to do geospatial search (using the location module, geofield, or other modules that stored latitude and longitude coordinates in Drupal that could be indexed by Apache Solr). Life was good—as long as you only had one location per node!

Sometimes, you may have a node (say a product, or a personality) affiliated with multiple locations. Perhaps you have a hammer that's available in three of your company's stores, or a speaker who is available to speak in two locations. When solr 3.x and Drupal 7 encountered this situation, you would either use a single location value in the index (so the second, third, etc. fields weren't indexed or searched), or if you put multiple values into solr's search index using the LatLonType, solr could throw out unexpected results (sometimes combining the closest latitude and closest longitude to a given point, meaning you get strange search results).

With Solr 4.x, especially Solr 4.3 and 4.5, there were some great improvements to spatial search, mostly enabled by switching from the old and trusty single-value LatLonType to the new (but slightly slower) SpatialRecursivePrefixTreeFieldType, or RPT for short.

Pages

Subscribe to RSS - drupal