Track and assembly hubs are collections of data that are hosted on your servers and can be displayed using the UCSC Genome Browser and other genome browsers supporting the UCSC track hub format. Track hubs allow for the visualization of data on assemblies that we already host (such as the human or mouse genomes), while assembly hubs can be used to create genome browsers for any genome assembly of your choosing.

Hubs depend on a number of different plain text configuration files. The most important are the trackDb.txt files for each assembly in your hub. These files contain the track configuration settings, also known as “trackDb settings”, that control how the track displays in the Genome Browser as well as the display of the item detail pages. You can see the trackDb settings available for hubs on the Hub Track Database Definition page.

As the track hub format has grown in popularity, other genome browsers, including Ensembl, Biodalliance, and the WashU Epigenome Browser, have implemented support for the UCSC track hub format. The Ensembl genome browser currently boasts fairly comprehensive support of the UCSC track hub format. In addition to supporting track hubs on their site, the Ensembl team has also created a Track Hub Registry that pulls hubs listed on our Public Hubs page into a centralized database alongside those hubs submitted to their registry. In an attempt to make the adoption of our track hub format easier, we talked to the other genome browsers about what settings were core to a track hub being, well, a track hub. We sort the list of hundreds of settings into various support levels, which include:

• Required – needed to display a hub across the different browsers.
• Base – non-required settings that are likely to be supported by other genome browsers
• Full – all other trackDb settings fully supported in the UCSC Genome Browser
• New – settings introduced since the last versioned release, may change between now and the next versioned release
• Deprecated – settings that may currently work, but could cease to work in the future as they are not being actively developed

We periodically increment the trackDb version number as major updates and changes are made to the settings. The latest change — version 2 — included settings related to the release of several “big*” file types, such as bigGenePred, bigPsl, bigChain, bigMaf, and CRAM. It also included moving several settings (html, priority, colorByStrand, autoScale, spectrum) from the “full” level to the “base” level to indicate that they are supported at other genome browsers (at this time primarily Ensembl).

During the initial versioning process, we improved the “hubCheck” utility to check the support of the trackDb settings used in your hub against our master list of trackDb settings, by version and support level. The hubCheck tool can be utilised in a variety of different ways; principally it checks if your hub works, but it can also list your hub’s settings and their support levels (required, deprecated, base, full and new) as well as check the support of your settings against any genome browser.  For example, to test compatibility with Ensembl (which supports the ‘base’ level of hub settings), use the command:

hubCheck -checkSettings -level=base http://genome.ucsc.edu/goldenPath/help/examples/hubDirectory/hub.txt

You can see more examples of how you might use hubCheck to check the compatibility of your hub with other genome browsers in our help documentation. To acquire hubCheck, you can click Downloads from the top blue menu bar and then select Utilities and navigate to the utilities directory.

If you have questions about creating or validating your track and assembly hubs, please feel free to contact us!

For more information on hubs in the UCSC Genome Browser, please see the following pages:

• Track Hub User Guide
• Hub Track Database Definition

For more information on hubs in other genome browsers, see their help pages here:

• Biodalliance
• Ensembl
• WashU Epigenome Browser

Questions about other genome browsers support for hubs should be directed to their mailing lists.

If after reading this blog post you have any public questions, please email genome@soe.ucsc.edu. All messages sent to that address are archived on a publicly accessible forum. If your question includes sensitive data, you may send it instead to genome-www@soe.ucsc.edu.

The release of the new NCBI RefSeq track marks a major shift in how we include annotations from NCBI’s Reference Sequence Database (RefSeq) in the UCSC Genome Browser. This new track is a composite track that contains the combined set of curated and predicted annotations from the RefSeq database for hg38/GRCh38. It also contains tracks that break up the annotation set into a few subsets. These subsets include only the curated transcripts (NM, NR, or YP transcripts), only the predicted transcripts (XM or XR transcripts), all of the other annotations from RefSeq that don’t fit into the curated or predicted subsets, and the alignments of the curated and predicted transcripts to the genome. All of the coordinates and alignments in these tracks are provided by the RefSeq group.

This new NCBI RefSeq composite also includes a “UCSC RefSeq” track that is based on our original method of producing the “RefSeq Genes” track. This “UCSC RefSeq” track is built by aligning RNAs obtained from the RefSeq Database to the genome. In the early days of the UCSC Genome Browser, only RNA sequences were provided by RefSeq, so we used BLAT to align them to the genome. This was a good solution in the past, but over time this method has led to some issues with transcripts matching to multiple places and our alignments of small exons or other regions differing slightly from those found in the RefSeq database. This type of minor alignment difference can be seen in the following session, where you can see that the RefSeq Curated (top) and UCSC RefSeq (bottom) tracks place the small fifth exon in transcript NM_001130970 at different locations due to the fact that there are multiple matches to this exon sequence in that region.

The new set of RefSeq tracks differs from the “UCSC RefSeq” track in a few key ways. First, as mentioned previously, the new tracks are based entirely on positions and alignments provided by RefSeq. Second, this track is currently only available for the hg38/GRCh38 assembly. This means that if you obtain the hg38 coordinates for a RefSeq transcript from the UCSC Genome Browser, these coordinates should be the same as those from the entry found at NCBI’s RefSeq Database. Lastly, these new NCBI RefSeq tracks include predicted transcripts, which were absent from our original RefSeq track.

This has been a long and exciting collaboration between the UCSC Genome Browser staff and NCBI’s RefSeq group. We trust that this full complement of tracks from the Reference Sequence Database will be helpful to you, our Browser users. We hope to bring these tracks to more genome assemblies in the future.

If after reading this blog post you have any public questions, please email genome@soe.ucsc.edu. All messages sent to that address are archived on a publicly accessible forum. If your question includes sensitive data, you may send it instead to genome-www@soe.ucsc.edu.

If you think dogs can’t count, try putting three dog biscuits in your pocket and then giving Fido only two of them.  

~Phil Pastoret

“Counting is easy. Right?”

I say this with my hand out, my thumb and 4 fingers spread out. With my other hand’s pointer finger, I simply count each digit, “one, two, three, four, five.” Easy.

But what happens when you start counting at 0 instead of 1? You can see that you have 5 digits (4 fingers and a thumb), but how do you calculate the size of your range?

With your hand in mind as an example, let’s look at counting conventions as they relate to bioinformatics and the UCSC Genome Browser genomic coordinate systems.

The UCSC Genome Browser uses two different systems:

Table 1. UCSC Genome Browser coordinate systems summary

Section 1: Interval types

0-start vs. 1-start : Does counting start at 0 or 1?
Synonyms:
Sometimes referred to as “0-based” vs “1-based” or “0-relative vs “1-relative.”

Interval Types
For a counted range, is the specified interval fully-open, fully-closed, or a hybrid-interval (e.g., half-open)?

Ok, time to flashback to math class!
You might recall that specifying an interval type as open, closed (or a combination, e.g., “half-open”) refers to whether or not the endpoints of the interval are included in the set. For further explanation, see theinterval math terminology wiki article. Figure 1 below describes various interval types.

Figure 1. (To enlarge, click image.) Description of interval types.

Section 2: Interval types in the UCSC Genome Browser

UCSC Genome Browser web interface = “1-start, fully-closed”

A common counting convention is a system that we all used when we first learned to count the fingers on our hands; this is referred to as the “one-based, fully-closed” system (Figure 2, below). Note that an extra step is needed to calculate the range total (5).

The “1-start, fully-closed” system is what you SEE when using the UCSC Genome Browser web interface. However, all positional data that are stored in database tables use a different system.

Figure 2. (To enlarge, click image.) 1-start, fully-closed interval. Most common counting convention. Used within the UCSC Genome Browser web interface (but not used in UCSC Genome Browser databases/tables). We calculate that we have 5 digits because 5 (pinky finger, range end) – 1 (the thumb, range start) = 4. We then need to add one to calculate the correct range; 4+1= 5.

UCSC Genome Browser tables = “0-start, half-open”

While the commonly-used “one-start, fully-closed” system is more intuitive, it is not always the most efficient method for performing calculations in bioinformatic systems, because an additional step is required to calculate the size of the base-pair (bp) range.

To increase efficiency, the UCSC Genome Browser uses a “hybrid-interval” coordinate system for storing coordinates in databases/tables that is referred to as “0-start, half-open” (see Figure 3, below).

Although coordinates in the web browser are converted to the more human-readable “1-start, fully-closed” system, coordinates are stored in database tables as “0-start, half-open.” You may have heard various terms to express this 0-start system:

Synonyms for “0-start, half-open”

• 0-based, half-open
• 0-based start, 1-based end
  • Note: This is not technically accurate, but conceptually helpful. A “1-based end” refers to the end of the range being included, as in the common “1-based, fully-closed” system.
• 0-start, hybrid-interval (interval type is: start-included, end-excluded)

Figure 3. (To enlarge, click image.) The UCSC Genome Browser coordinate system for databases/tables (not the web interface) is “0-start, half-open” where start is included (closed-interval), and stop is excluded (open-interval). We calculate that we have 5 digits because 5 (range end after pinky finger) – 0 (the thumb, range start)  = 5.

Another example which compares 0-start and 1-start systems is seen below, in Figure 4. This figure describes the differences in defining and calculating the range for a specified sequence highlighted in yellow, “T, C, G, A.”

Figure 4. (To enlarge, click image.)  Calculation of genomic range for comparing “1-start, fully-closed” vs. “0-start, half-open” counting systems.

Section 3: Formatting

Coordinate formatting indicates interval type

The UCSC Genome Browser and many of its related command-line utilities distinguish two types of formatted coordinates and make assumptions of each type.

The “Position” format (referring to the “1-start, fully-closed” system as coordinates are “positioned” in the browser)

• Written as: chr1:127140001-127140001
• No spaces.
• Includes punctuation: a colon after the chromosome, and a dash between the start and end coordinates.
• When in this format, the assumption is that the coordinate is 1-start, fully-closed.

The “BED” format (referring to the “0-start, half-open” system)

• Written as: chr1 127140000 127140001
• Spaces between chromosome, start coordinate, and end coordinate.
• No punctuation.
• When in this format, the assumption is that the coordinates are 0-start, half-open.

Section 4: Examples

SNP example

What we SEE in the Genome Browser interface itself is the “1-start, fully-closed” system. However, these data are not STORED in the UCSC Genome Browser databases and tables in the same way. The UCSC Genome Browser databases store coordinates in the “0-start, half-open” coordinate system.

Table 2. SNP coordinates in web browser (1-start) vs table (0-start)

LiftOver examples and coordinate formatting

Let’s take a look at the two types of coordinate formatting (“BED” and “position”) when using the UCSC Genome Browser web-based and command-line utility liftOver tools.

1) Web-based LiftOver example

Below is an example from the UCSC Genome Browser’s web-based LiftOver tool (Home > Tools > LiftOver). Depending on how input coordinates are formatted, web-based LiftOver will assume the associated coordinate system and output the results in the same format.

Table 3. UCSC Genome Browser web-based LiftOver and “position” coordinate formatting

Table 4. UCSC Genome Browser web-based LiftOver and “BED” coordinate formatting

 * Note that the web-based output file extension is misleading in this case; while titled “*.bed” the positional output is not actually in “0-start, half-open” BED format, because the 1-start, fully-closed “positional” format was used for input. 

 2) Command-line liftOver utility example

When using the command-line utility of liftOver, understanding coordinate formatting is also important. Just like the web-based tool, coordinate formatting specifies either the “0-start half-open” or the “1-start fully-closed” convention. For example, if you have a list of 1-start “position” formatted coordinates, and you want to use the command-line liftOver utility, you will need to specify in your command that you are using “position” formatted coordinates to the liftOver utility.

To view the liftOver utility usage statement and options, enter “liftOver” on your command-line (with no other arguments, and without the quotes).

Table 5. UCSC Genome Browser command-line liftOver and “position” coordinate formatting

Table 6. UCSC Genome Browser command-line liftOver and “BED” coordinate formatting

Wiggle Files

The wiggle (WIG) format is used for dense, continuous data where graphing is represented in the browser. Wiggle files of variableStep or fixedStep data use “1-start, fully-closed” coordinates. Like all other UCSC Genome Browser data, these coordinates are positioned in the browser as “1-start, fully-closed.”

Note: Many other formats outside of the UCSC Genome Browser use 1-start coordinate systems, such as GTF/GFF.

Table 7. UCSC Genome Browser wiggle files & coordinate systems

 Section 5: Resources

• Sequence Coordinates: 0- vs 1-base, Bob Milius, PhD [pdf]
• Cheat Sheet For One-Based Vs Zero-Based Coordinate Systems [Biostars Forum]
• Database/browser start coordinates differ by 1 base [UCSC Genome Browser, FAQ]
• Genome wiki: Coordinate Transforms [UCSC Genome Browser Wiki: “genomewiki”]
• UCSC Genome Browser: wiggle format help page
• Kent WJ, Zweig AS, Barber G, Hinrichs AS, Karolchik D. BigWig and BigBed: enabling browsing of large distributed data sets. Bioinformatics. 2010 Sep 1;26(17):2204-7. Epub 2010 Jul 17.

If after reading this blog post you have any public questions, please email genome@soe.ucsc.edu. All messages sent to that address are archived on a publicly accessible forum. If your question includes sensitive data, you may send it instead to genome-www@soe.ucsc.edu.

Have you been wondering when we’ll get some of that next-gen gene expression in human tissues up as tracks in the browser? The GNF Atlas microarray tracks are so 2004… Yes, we do have RNA-seq from ENCODE cell lines, but you can get only so far with cell lines (are they even human?). Well, wait no longer! Once we learned what the GTEx folks are up to – RNA-seq and genotyping of samples from 53 tissues in many hundreds of donors – we just had to get on board! Read on for details…

The NIH Genotype-Tissue Expression (GTEx) project was created to establish a sample and data resource for studies on the relationship between genetic variation and gene expression in multiple human tissues. In April this year the Genome Browser released the GTEx Gene Expression track, which showcases data from the GTEx midpoint milestone data release (V6, October 2015) – 8555 tissue samples obtained from 570 adult postmortem individuals. The track shows median expression level per tissue at each gene via a new bar graph display:

The height of each bar represents the median expression level across all samples for a tissue, and the bar color indicates the tissue (we are using GTEx publication color conventions). You can see the gene description and tissue name with expression level when you mouseover, and can view the tissue legend in glorious detail on the track configuration page. Above, notice the 3 highly expressed tissues for TCAP protein (titin-cap, used in muscle assembly) – unsurprisingly in this case, heart (2 sub-tissues) and skeletal muscle.

In the tissue mix sampled by GTEx, you’ll find a dozen brain sub-tissues, a handful of cardiovascular tissues, and bits from digestive, reproductive, and endocrine systems. For a nice summary of the tissues assayed, check out the GTEx project portal. Not so interested in all the tissues? Turn on the tissue filter and limit the graph to show just your faves!

Once you’ve found your favorite gene, you can drill down for more detail. A nice boxplot showing the range for all samples and the sample count is right here on the details page:

You’ll also see this plot on the new RNA-Seq Expression panel of the UCSC Genes detail page:

If gene-level calls aren’t your thing – you’re more of a deep diver and want to see the actual RNA-seq coverage – you might find the newly released GTEx Signal Hub just your style. We were fortunate to be able to team up with the Global Alliance crowd here within the UCSC Genomics Institute and convince them to pump all the available GTEx RNA-seq through their hot new Toil pipeline (along with twice as much cancer data) to produce signal graphs. A round of ‘biggification’, lifting and track configuration (gotta have those GTEx colors!) produced the hub. Find it on the Public Hubs panel of the Track Hubs page, which you can navigate to via the My Data > Track Hubs menu option in the top blue bar.

Did I mention you can find the GTEx gene track and the GTEx Signal hub on both the hg19 (GRCh36) and hg38 (GRCh37) genome browsers?

Give the new tracks a spin! To get you started, here’s a session:

Now enjoy!!

If after reading this blog post you have any public questions, please email genome@soe.ucsc.edu. All messages sent to that address are archived on a publicly accessible forum. If your question includes sensitive data, you may send it instead to genome-www@soe.ucsc.edu.

New UCSC Genome Browser gateway page design

The opinions expressed here are those of the author, Cath Tyner, and do not necessarily reflect those of the University of California Santa Cruz or any of its units.

Maybe it’s just me, but I can clearly remember the excitement of getting brand new sparkly shoes as a young kid.  Half the excitement was picking out the shoes – my siblings and I would try on every potential new-shoe option. There were high standards, of course; rigorous criteria that had to be thoroughly discussed and tested. Could they make you jump SUPER high? Low tops or high tops? Classic laces or cutting edge velcro? After finally picking out the perfect pair and racing to put them on with maniacal laughter, the reality set in. New shoes. Ahhhh. The excitement of showing off my new kicks at school was only one night away.

That’s a little bit how we all felt when we unveiled the brand new sparkly Genome Browser gateway page  earlier this week. This was a project that had been “in the works” for quite a long time, starting from ideas and drawings, moving into design phases, and finally maturing into many iterations of testable versions as the development process gained its own momentum. This project soon had a life of its own – we all became shepherds as we guided it into what we finally knew was a final product.

The things we are most excited about? We’ve already received feedback that the new human-centric phylogenetically ordered tree menu is downright awesome (and we think so too).  For me, the graphics and colors pull me in, inviting me to visually scroll through our entire genome species collection. With a flick of the scroll handle on the tree menu, I can zip from “us humans” all the way down to sea hare or Ebola virus; within two seconds, I’ve just traveled through millions and millions of evolutionary years. Based on NCBI’s taxonomy database, the “tree menu” provides an interactive way to explore our genome species collection. Little known fact: Try hovering over one of the “branches” of the tree (the horizontal and vertical lines connecting all species) and see what you find!

Example of mouse hover on tree menu branch

Another exciting new feature that makes our eyes light up is the autocomplete search function and “popular species” button shortcuts:

Button shortcuts & autocomplete search

We know that over 95% of you will benefit from our “popular species” buttons as quick access shortcuts to the genomes that you use most. We also believe that just about everyone will benefit from the autocomplete search function. For example, you can enter “fish” to see genomes from our aquatic friends, or you can enter something as specific as “hg38” to load a particular assembly version. With a whopping 276 genomes and counting, autocomplete search is a celebrated new feature! The same autocomplete function works great for our public genome hubs; try typing “plant” to see related hubs.

Want to jump to your favorite gene in the genome browser? The “position/search term” functionality remains just as efficient – just enter a genomic position, gene symbol, or search term, lean back in your comfy chair, and press “GO.” You’re there.

To see more details, including a few menu option changes, visit the gateway announcement on our news page and watch the short gateway video tour.

We sincerely hope you enjoy the new gateway page as much as we do – and as always, we invite you to contact us with questions, concerns, and compliments. 😉

If after reading this blog post you have any public questions, please email genome@soe.ucsc.edu. All messages sent to that address are archived on a publicly accessible forum. If your question includes sensitive data, you may send it instead to genome-www@soe.ucsc.edu.

Genome Browser screen shot of the GRCh38 (hg38) human assembly showing the GENCODE Basic track opened in the PTEN region on chromosome 10.

As of Monday, July 29, 2015, the UCSC Genome Browser will use the GENCODE v22 comprehensive gene set as its default gene set on the human genome assembly GRCh38 (hg38), replacing the previous default set of genes created here at UCSC using code written by Jim Kent. This track, which is labeled as “GENCODE Basic” in the Genes and Gene Predictions track group, replaces UCSC Genes track as the default gene set.  We’re making this change in recognition of the value of reducing the number of competing gene sets used by the bioinformatics community.  With this change we will be using the same set of genes as Ensembl, reducing the potential for confusion, especially in clinical settings.

We’ve kept the same familiar UCSC Genes schema for the new gene set, using nearly all the same table names and fields that appeared in earlier versions of UCSC Genes. Hopefully this will make the transition to the new GENCODE models easier. Every transcript in the new set has both a UCSC ID and a GENCODE transcript ID. There are a couple of new tables: knownCds, which has the coding frame numbers for each gene, and knownToMrna, which captures the association to GenBank mRNAs. A couple tables are no longer present: knownGeneTxMrna and knownGeneTxPep.

By default, we display only the transcripts tagged as “basic” by the GENCODE Consortium. However, all the transcripts in the GENCODE comprehensive set are present in the tables. You can view them in the browser by selecting “show comprehensive set” in the “Show” section of the track’s description page. On that same page, you can also configure the browser to label the genes with the GENCODE transcript IDs by selecting “GENCODE Transcript ID” label option.

The new gene set has 195,178 total transcripts, compared with 104,178 in the previous UCSC Genes version. The total number of canonical genes, now defined using the GENCODE gene loci ( ENSG* identifiers), has increased from 48,424 to 49,534.

Comparing the previous gene set with the new version:

• 9,459 transcripts are identical.
• 22,088 transcripts were not carried forward to the new version.
• 43,681 have consistent splicing, but changes in the UTR.
• 28,950 transcripts overlap with those in the previous set, but have
  at least one different splice.

We plan to continue using the previous UCSC computational pipeline to generate the default gene set on the mouse assembly, GRCm38 (mm10), for the foreseeable future. We will also periodically update the old UCSC-computed gene set on the human GRCh38 assembly as an ancillary track (“Old UCSC Genes”) without the rich set of link-outs we maintain for the default gene set.

If after reading this blog post you have any public questions, please email genome@soe.ucsc.edu. All messages sent to that address are archived on a publicly accessible forum. If your question includes sensitive data, you may send it instead to genome-www@soe.ucsc.edu.