Synteny from liftOver chains (hg38 and eight vertebrates)

JBrowse v5 beta

The tutorials target the JBrowse v5 beta, and the v4.3.0 release on the download page lacks some of what they show. To install the beta of JBrowse Web, run npm install -g @jbrowse/cli@next, then jbrowse create jbrowse2 --branch v5.0.0-beta.11. Desktop beta builds are coming soon.

We look at one human locus across eight other mammals at once, from alignments that already exist. UCSC publishes a liftOver chain from hg38 to every genome it hosts, and jbrowse.org keeps each of those chains as an indexed alignment file, so one track composes eight of them into a lane per genome under the human view, each lane drawing the RefSeq gene models from the UCSC hub for that genome. At TP53 most ape lanes come out nearly continuous and mouse, dog and cow show where their chains gap; eight megabases away every lane breaks and most reverse, and the page ends by reading one of those breaks back out of its chain file.

Prerequisites

htslib (tabix), to read a hosted index without downloading the alignment
A running JBrowse instance (the web quickstart or the desktop quickstart)

Where the data comes from

Show full URLs

UCSC's pairwise liftOver chains from hg38 (Kent et al. 2003), each converted to an indexed PAF and rehosted beside the hub configs genomes.jbrowse.org serves, and eight UCSC genome hubs, whose assembly entry and NCBI RefSeq gene track each lane takes verbatim.

The chains

UCSC's liftOver folder for hg38, one hg38To<Genome>.over.chain.gz per genome: hgdownload.soe.ucsc.edu/…/liftOverhttps://hgdownload.soe.ucsc.edu/goldenPath/hg38/liftOver/

The same chains as indexed alignments

The hub configs

hg38: jbrowse.org/…/config.jsonhttps://jbrowse.org/ucsc/hg38/config.json
every other genome at the same path under its UCSC name, panTro6 for one: jbrowse.org/…/config.jsonhttps://jbrowse.org/ucsc/panTro6/config.json
the finished config, the eight hub entries and the composed track together: jbrowse.org/…/config.jsonhttps://jbrowse.org/demos/hg38_vertebrates/config.json

LiftOver chains as one indexed alignment per genome

A liftOver chain maps each stretch of one genome to a position in another, whether or not a gene lies there, so the track can draw each lane from the chain alone.

Each UCSC chain aligns hg38 to one other genome, so the set forms a star with hg38 at the centre: hg38 against chimpanzee, hg38 against mouse, and no row that aligns chimpanzee to mouse. The track places each lane from the chain between hg38 and that genome, and draws the ribbon between two neighbouring lanes through the human coordinates both share.

Each hosted file is a chain converted to PAF and indexed with jbrowse make-pif, which can write a coarser tier beside the per-base one for whole-chromosome zooms. tabix lists the tiers from the index without downloading the alignment: the header names them, and coarse-tier sequence names are upper-case.

To index a chain of your own, convert it to PAF with chain2paf from paftools, then run make-pif:

paftools.js chain2paf genomeA.genomeB.over.chain.gz > genomeA_genomeB.paf
jbrowse make-pif genomeA_genomeB.paf

# the #pif header, if the file was built recently enough to have one
tabix -H https://jbrowse.org/ucsc/hg38/liftOver/hg38ToPanTro6.over.pif.gz | awk 'NR==1'
# how many coarse-tier sequences the index holds; zero means one tier
tabix -l https://jbrowse.org/ucsc/hg38/liftOver/hg38ToPanTro6.over.pif.gz | grep -c '^[TQ]'

Composing the eight chains into one track

One SyntenyTrack names hg38 and every genome it stacks. Its adapter, a MultiPairwiseSyntenyAdapter, holds one PairwiseIndexedPAFAdapter per chain, each naming its two assemblies as query,target with the genome the chain lifts to first. The anchor is the one assembly every chain names. The config below is cut to three genomes for the page, and the hosted config has all eight. The assemblies and their gene tracks come from the hub configs unchanged, and each lane finds its gene models through the session, so this track is the only addition to the hub entries. A genome of your own loads from its FASTA instead:

Config file

Goes in the assemblies array of config.json. See Assemblies.

{
  "name": "myGenome",
  "uri": "myGenome.fa.gz"
}

CLI (add-assembly)

jbrowse add-assembly myGenome.fa.gz \
  --name myGenome \
  --load copy

JBrowse Desktop

In JBrowse Desktop, Open new genome on the start screen (or File → Open genome... in a session), then Open from a URL and paste, one per line:

myGenome.fa.gz
myGenome.fa.gz.fai
myGenome.fa.gz.gzi

JBrowse reads the format off the file name. Then fill in:

Genome name: myGenome

myGenome.fa.gz is relative to a config.json. Replace it with its URL or its path on this computer.

Config file

Goes in the tracks array of config.json. See Tracks.

{
  "type": "SyntenyTrack",
  "trackId": "hg38_liftover_multiway",
  "name": "hg38 vs 8 UCSC genomes (liftOver chains)",
  "assemblyNames": ["hg38", "panTro6", "rheMac10", "mm39"],
  "adapter": {
    "type": "MultiPairwiseSyntenyAdapter",
    "adapters": [
      {
        "type": "PairwiseIndexedPAFAdapter",
        "uri": "https://jbrowse.org/ucsc/hg38/liftOver/hg38ToPanTro6.over.pif.gz",
        "csi": true,
        "assemblyNames": ["panTro6", "hg38"]
      },
      {
        "type": "PairwiseIndexedPAFAdapter",
        "uri": "https://jbrowse.org/ucsc/hg38/liftOver/hg38ToRheMac10.over.pif.gz",
        "csi": true,
        "assemblyNames": ["rheMac10", "hg38"]
      },
      {
        "type": "PairwiseIndexedPAFAdapter",
        "uri": "https://jbrowse.org/ucsc/hg38/liftOver/hg38ToMm39.over.pif.gz",
        "csi": true,
        "assemblyNames": ["mm39", "hg38"]
      }
    ]
  },
  "displays": [
    {
      "type": "MultiWaySyntenyDisplay",
      "height": 600
    }
  ]
}

CLI (add-track-json)

jbrowse add-track-json '{
  "type": "SyntenyTrack",
  "trackId": "hg38_liftover_multiway",
  "name": "hg38 vs 8 UCSC genomes (liftOver chains)",
  "assemblyNames": ["hg38", "panTro6", "rheMac10", "mm39"],
  "adapter": {
    "type": "MultiPairwiseSyntenyAdapter",
    "adapters": [
      {
        "type": "PairwiseIndexedPAFAdapter",
        "uri": "https://jbrowse.org/ucsc/hg38/liftOver/hg38ToPanTro6.over.pif.gz",
        "csi": true,
        "assemblyNames": ["panTro6", "hg38"]
      },
      {
        "type": "PairwiseIndexedPAFAdapter",
        "uri": "https://jbrowse.org/ucsc/hg38/liftOver/hg38ToRheMac10.over.pif.gz",
        "csi": true,
        "assemblyNames": ["rheMac10", "hg38"]
      },
      {
        "type": "PairwiseIndexedPAFAdapter",
        "uri": "https://jbrowse.org/ucsc/hg38/liftOver/hg38ToMm39.over.pif.gz",
        "csi": true,
        "assemblyNames": ["mm39", "hg38"]
      }
    ]
  },
  "displays": [
    {
      "type": "MultiWaySyntenyDisplay",
      "height": 600
    }
  ]
}'

JBrowse Desktop

In JBrowse Desktop, or in any running JBrowse Web session, open a view on this track’s assembly, then File → Open track..., choose Add track from pasted JSON, and paste:

{
  "type": "SyntenyTrack",
  "trackId": "hg38_liftover_multiway",
  "name": "hg38 vs 8 UCSC genomes (liftOver chains)",
  "assemblyNames": ["hg38", "panTro6", "rheMac10", "mm39"],
  "adapter": {
    "type": "MultiPairwiseSyntenyAdapter",
    "adapters": [
      {
        "type": "PairwiseIndexedPAFAdapter",
        "uri": "https://jbrowse.org/ucsc/hg38/liftOver/hg38ToPanTro6.over.pif.gz",
        "csi": true,
        "assemblyNames": ["panTro6", "hg38"]
      },
      {
        "type": "PairwiseIndexedPAFAdapter",
        "uri": "https://jbrowse.org/ucsc/hg38/liftOver/hg38ToRheMac10.over.pif.gz",
        "csi": true,
        "assemblyNames": ["rheMac10", "hg38"]
      },
      {
        "type": "PairwiseIndexedPAFAdapter",
        "uri": "https://jbrowse.org/ucsc/hg38/liftOver/hg38ToMm39.over.pif.gz",
        "csi": true,
        "assemblyNames": ["mm39", "hg38"]
      }
    ]
  },
  "displays": [
    {
      "type": "MultiWaySyntenyDisplay",
      "height": 600
    }
  ]
}

That shows it in the linear view. For the synteny view, open Add → Linear synteny view, pick the track under Quick start, and click Launch.

Level of detail on the track menu picks which stored tier a zoom reads, as on a pairwise synteny track. The menu offers it once every child has a coarse tier; run the tabix -l line above on each file to check.

Stacking nine genomes at TP53 and at PMP22

Opened on hg38 around TP53, the track draws a lane per genome under the human axis, each fitted to wherever its chain places the window. Each lane header names the chromosome and the span the lane shows, with [rev] where that chromosome runs the other way relative to hg38. The lanes stack with the genomes placing the most of the window at the top.

Config file

Goes at the top level of config.json, replacing any defaultSession there. See Default session.

{
  "defaultSession": {
    "name": "TP53 neighbourhood across nine vertebrates",
    "views": [
      {
        "type": "LinearGenomeView",
        "assembly": "hg38",
        "loc": "chr17:7,400,000-7,700,000",
        "tracks": [
          "hg38-ncbiRefSeq",
          {
            "trackId": "hg38_liftover_multiway",
            "type": "MultiWaySyntenyDisplay",
            "height": 600
          }
        ]
      }
    ]
  }
}

CLI (set-default-session)

jbrowse set-default-session --session - << 'EOF'
{
  "name": "TP53 neighbourhood across nine vertebrates",
  "views": [
    {
      "type": "LinearGenomeView",
      "assembly": "hg38",
      "loc": "chr17:7,400,000-7,700,000",
      "tracks": [
        "hg38-ncbiRefSeq",
        {
          "trackId": "hg38_liftover_multiway",
          "type": "MultiWaySyntenyDisplay",
          "height": 600
        }
      ]
    }
  ]
}
EOF

Open in JBrowse

Open this session in JBrowse Web ↗

Around TP53 every lane but gorilla places the whole window from one chain, with the same genes in the same order. The gorilla lane stops after FXR2, because gorGor6 holds the TP53 end of the window on an unplaced contig. A chain records its insertions and deletions inside one alignment record, and the track cuts a record at every indel of 10 kb or more, drawing one ribbon per gap-free run. The white wedges between ribbons are stretches one genome has and the other lacks. The mouse, cow and dog chains have many such gaps, and the ape chains have gaps under the cut, so their ribbons look continuous. The mouse lane shows [rev] because its chain runs the other way against hg38 here.

The TP53 neighbourhood on hg38 over eight UCSC genome lanes, each drawing its own RefSeq gene models. White wedges in the ribbons are large indels in each chain, few in the apes and many in mouse, cow and dog. The mouse lane is reversed, and the gorilla lane stops where gorGor6 moves the rest of the window onto an unplaced contig.

Navigate to chr17:15,200,000-16,400,000, eight megabases toward the centromere, near PMP22. The region is a hotspot of segmental duplications (long repeated copies of sequence): every genome places it as several blocks, and many of those blocks run the other way against their neighbours.

A lane whose blocks mostly run backwards along hg38 shows [rev] in its header and draws mirrored, so its inversions come out straight. Inside a lane, a ribbon crosses where one block runs against its neighbour. The marmoset lane is the clearest case: its chain covers the left of the window forwards and then stops, and a reversed block further along the same chromosome places the rest of the window.

The gorilla lane header names chr17 beside chr5, the chromosome the lane drew. Part of the window aligns to chr17, the homologous chromosome, so the chr5 frame misses some of what the alignment places. Show chr17 in this lane on the lane header menu pins the lane onto chr17. Where a lane's blocks split about evenly between the two orientations, Flip lane on its header menu turns the lane the other way on the same contig, and Let the lane choose its orientation hands the choice back.

hg38 chr17 near the PMP22 segmental duplications over the same eight lanes. A lane running backwards along hg38 carries a reversed marker in its header, crossed ribbons mark a block running against its neighbour inside a lane, and the gorilla lane names a second chromosome that also places the window.

Color by... → Strand, under Ribbons on the track menu, colors each ribbon by strand. A ribbon between the anchor and the first lane takes the strand of one alignment, and a ribbon between two genome lanes takes the product of two alignments' strands, so two reverse alignments give forward. The track flips a lane whose alignments all run the other way and marks it [rev] in its header, so its ribbons come out straight on screen while the strand color still marks each of them as an inversion. A single crossed ribbon into an unflipped lane is one block running against its neighbours.

The eight lanes at chr17 near the PMP22 duplications, ribbons colored by strand. In a flipped lane the ribbons run straight and still carry the reverse color.

Reading a lane's span, width multiple and ribbons

Each lane draws in the coordinates of its own genome, fitted to wherever the alignment places the hg38 window. The number at a lane's right edge is the span the lane shows, and the multiple after it is how many times wider than the hg38 window that span is, rounded to one of a few fixed steps. A lane that places up to a tenth more than a step stays on that step, and the extra runs off the lane's edges.

A ribbon joins a lane to the lane directly above it. No chain aligns two neighbouring genomes in this star, so the track composes their ribbon through hg38, the anchor. Hovering a ribbon lights the same alignment in every lane it reaches, dragging a lane label reorders the stack, and a lane's header menu re-anchors the view on that genome or opens it in a separate view.

Checking the marmoset lane against its chain

The marmoset rows over the PMP22 window come straight out of the hosted index:

# marmoset rows over the window, 50 kb and longer:
# hg38 start, end, strand, marmoset contig, start, end
tabix https://jbrowse.org/ucsc/hg38/liftOver/hg38ToCalJac4.over.pif.gz tchr17:15200000-16400000 \
  | awk -F'\t' '$11 > 50000' | cut -f3,4,5,6,8,9

The marmoset rows come back on both strands of chr5, the forward chain the lane follows and the reversed block it draws as a crossing.

Reproduce it end to end

The script writes one config, with nothing to build or upload, in three steps:

Probe which of the listed genomes have both a hub config and an indexed chain with its .csi on jbrowse.org, and drop the rest.
Report the tiers each chain file holds, since Level of detail needs a coarse tier in every child.
Write each kept genome's hub entry and gene track, and one composed track with a child per chain.

GENOMES picks a different set; see Prerequisites.

curl -fO https://raw.githubusercontent.com/GMOD/jbrowse-components/main/scripts/build_hg38_liftover_multiway.sh
bash build_hg38_liftover_multiway.sh   # writes ./hg38_vertebrates_build/config.json

External links

UCSC Genome Browser downloads: https://hgdownload.soe.ucsc.edu/downloads.html

Citations

Kent WJ, Baertsch R, Hinrichs A, Miller W, Haussler D. Evolution's cauldron: duplication, deletion, and rearrangement in the mouse and human genomes. Proc Natl Acad Sci USA (2003). https://doi.org/10.1073/pnas.1932072100

Feedback on this tutorial is welcome: contact us.

Synteny from liftOver chains (hg38 and eight vertebrates)

Prerequisites#

Where the data comes from#

LiftOver chains as one indexed alignment per genome#

Composing the eight chains into one track#

Stacking nine genomes at TP53 and at PMP22#

Reading a lane's span, width multiple and ribbons#

Checking the marmoset lane against its chain#

Reproduce it end to end#

See also#

External links#

Citations#

Prerequisites

Where the data comes from

LiftOver chains as one indexed alignment per genome

Composing the eight chains into one track

Stacking nine genomes at TP53 and at PMP22

Reading a lane's span, width multiple and ribbons

Checking the marmoset lane against its chain

Reproduce it end to end

See also

External links

Citations