For the last few months, a team of geography students at SDSU have been working with the crime data provided by the Library, producing analyses and visualizations of the data. Elias Issa has been looking at Drugs and Alcohol violations in Downtown San Diego and East Village. He writes:

The Hot Spot tool calculates the Gi* statistic for each feature in a dataset. The resultant z-scores and p-values tell you where features with either high or low values cluster spatially. To have a statistically significant hot spot, a feature will have a high value and be surrounded by other features with high values as well. My animated maps illustrate 2 hot spots from 2007 to 2012. the most significant hot spot is located close to St. Vincent Paul homeless shelter ( Imperial Ave) on the South Eastern part of East Village. The second hot spot is located North of Gaslamp Quarter between Broadway And Market which most of the famous and popular bars are found. In addition to those map,and based on those hot spots, I did some statistical analysis to show the average of Drugs/ Alcohol violation monthly (Above/ Below Avg) and yearly. My study reveals that there is a slight increase within 2011 and 2012 in the average of Drugs/ Alcohol violations.

SANDAG, through its public safety division ARJIS, is now publishing crime data to the web. This is a major advance in accessibility, since previously crime incidents were only available through a Public Records Act request, and usually involved a fee. The download file includes crime incidents for the last 180 days. The Library will be archiving these files occasionally, and adding them to our crime incident datasets. The files are updated weekly. Unlike the data we acquired previously, this release does not include the ’legend’ field, which is the high level category for the type of incident. As a result, we can’t incorporate these files into our previous releases without doing a lot of work to re-construct the ’legend’ field, so it may take a while before our datasets are getting updated.

We recently converted the SWITRS database of traffic collisions in California, extracting the records for San Diego County and creating a basic visualization in Tableau Public. Tableau Public is a fantastic data analysis tool, although it takes a bit of training to do complex things. Below is a simple visualization of the number of people killed and injured in San Diego county traffic collisions by day of week and hour of day, for the years 2002 to 2012, inclusive. The deaths line (orange) shows a familiar “bathtub” shape, with higher values on weekends and at night. The injured line is quite different. There are a lot of possibilities for this, including statistical effects due to the 100x difference in the number of injuries and deaths, or the qualitative differences between pedestrians at night during those of the day. For instance, it may be that injuries during the day are driven by travel patterns and reduced attentiveness in the afternoon, while fatalities are driven by alcohol consumption. Fortunately, there is enough information in the SWITRS dataset to test some of these hypothesis, but that will be the subject of another post.

We’ve released an update to the crime dataset today. This update is based on reprocessed data from SANDAG and includes some new fields.

San Diego Crime Incidents, Revision 3

This update is based on a new extract of the data we got recently from SANDAG. They didn’t tell me what had changed, just that they found some problems and re-ran the extract. If you want to compare the inputs, we have both the first revision and second revision of the SANDAG input data in the data repository. This release includes several new fields in the CSV and Shapefiles, including

What if you got really creative with data visualization and represented the street lights of La Jolla and Pacific Beach with the balloons that are used to make animals? It might like like this: An image of a dataset showing the distance, in centimeters, to the nearest street lamp.   This image is actually from a dataset I’ve created showing, for every 10m^2 area of the City of San Diego, how far the point in the middle of the area is from a street lamp. The image is colored so white is 0m and black is about 141m from the lamp. We’ll be using this data in our Crime Analysis Project to explore the connect between street lamps and crime.

Today we are releasing the processed version of six years of San Diego County crime data. We received this data last month from SANDAG, and have augmented the source data with city and and neighborhood codes. The data is released as both CSV and ESRI Shape files with one file per year. To download the data, visit our data repository. Be sure to read the documentation by expanding the pull-down bar at the top of the page. The improvements of this dataset over the original SANDAG data release include:

A few weeks ago, we received crime incident data from SANDAG for the years from 2007 to March of 2013. The data has the location of the crime listed as a “hundred block,” so a crime at 1435 Main street would be listed at 1400 Main. Since we don’t know where the crime occurred on the block - we don’t know the exact address - it makes sense to attribute the crime to the whole block. I just finished an initial pass at geocoding crimes to street segments, using the All_Roads file from SANGIS and lot of custom Python code. Here are the results: All crime incidents for 2007 to 2013, colored according to linear density on the street segment. In this map, all of the crime incidents for the 6 year period are counted, grouped by the block where the crime occurred. Then, the counts are divided by the length of the street segment and the time period, so the final value is Crime per year per kilometer. Then, the range of values is broken into 5 groups, using the Jenks natural breaks. ( Using head/tail breaks would require more programming ) I tweaked the breaks manually to put more segments into the top category. This presentation has an advantage over the previous maps we’d created, using Kernel Density Estimation to produce heatmaps: it is easier to see where high-crime blocks are. For instance, lets look at the red block east of downtown, due south of Balboa park. A crack house, perhaps? Nope, it is the Police station. It is very common for crimes to have their address set to the police station if it isn’t clear what an appropriate address would be, such as for a car chase or someone caught in a canyon, like in this hotspot in Mission Gorge:   It isn’t likely that the particular segment of road gets a lot of crimes, and the criminals probably don’t live in the mobile home park. That segment of road probably contains the address number that is used for crime committed in Mission Gorge, it maybe it is where the cops like to park to patrol the canyon. Here is a real hotspot, an empty lot that seems to be popular for drug use: Compare this to the heatmap view, zoomed out to show the contour of the hotspot: The heatmap works better at small scales, because you can quickly assess an entire region. At large scales, the street segment view can help isolate what a likely cause of the problem is, which can obscured by the large regions of the heatmap hotspots. The heat map also encourages a subtle psychological bias: the larger areas seem more significant, because our brains place a lot of emphasis on size. But in the segment map, the segments are viewed discretely, as qualities, not extents. So, it is easier to pay attention to unusual high intensity blocks which pop up because they are bright red, but which in the heatmap get ignored because they are small. We’re not quite ready to publish the processed crime data, but should have it released, along with both kinds of maps, in the next week.

Here is some eye candy, a population density map of Pacific Beach and surrounding neighborhoods.

This map was created with a lot of Python code, using the 2010 census shapefiles for census blocks, setting a value for each block as the population of the block divided by the area of the block, and rasterizing all of the blocks to an image. Red indicates areas of higher population density. You can clearly pick out the areas in Pacific Beach that are zoned for apartments vs single family homes, the UTC high-rise apartment area, and many other variations in land use.

Our resident data slinger sent me an initial analysis of time series data for specific crime hot spots in San Diego, and the results are interesting. Rather than look at the whole of the region at once, we’ve identified specific high intensity areas, and are doing time-series analysis for them independently. The major areas are: North Park, Chula Vista, Gaslamp and Pacific Beach. Since each of these areas is very different, they have different crime profiles, so we’re analyzing them separately. Here is the map for the Chula Vista analysis area: