Overview

Get started viewing your own data:

1. Install a Local Xena Hub on your computer

2. Start the Local Xena Hub

3. Load the data you want to view

4. View the data

We support most types of genomic and/or phenotypic/clinical/annotation data. Genomic data needs to be values called on genes, transcripts, exons, probes or some other identifier. Phenotypic/clinical/annotation data can be almost anything, including patient data (e.g. age, set, etc), clinical data (survival data for a KM plot), and other data such as gene fusion calls, regulon activity, immune scores, and more. Samples can be bulk tissue, cell lines, cells, and more. We do not visualize raw data such as FASTQs or BAMs.

Data can be your own or from another source, like GEO or a publication.

We support tab-delimited (.tsv and .txt) and Microsoft Excel files (.xlsx and .xls). Data on a Local Xena Hub can only be viewed or accessed by the same computer on which it is running, keeping private data secure.

The Local Xena Hub must be installed and running in order to load data, as well as any time you want to view data. The Local Xena Hub will remember previously loaded data.

Installing a Local Xena Hub

Click on VIEW MY DATA. You will be prompted to download and install a local Xena Hub.

Double click on the download to begin the installation of the Xena Hub. Follow the wizard to finish the install.

System requirements for Xena Hub

• Mac: OSX 10.7 and above

• Windows: 64-bit

• Linux: ability to run a .jar file

Starting/running a Local Xena Hub

After installing a local Xena Hub, go back to VIEW MY DATA to auto-start the Hub. If it does not automatically start, refresh the page or double click on the Xena Hub application on your computer. The Xena Hub application should be in your Applications folder for Mac and Windows. Note that it will take up to one minute to start up.

Loading data into a Local Xena Hub

Most people load data into their Local Xena Hub through our website wizard, which leads you through the loading process step by step. Note that you will want to make sure your data is properly formatted ahead of time.

You can also load data via the command line.

Viewing data from a Local Xena Hub

Click on VISUALIZATION. If your study is not already selected as step 1 of the wizard, then select it from the drop down and click 'Done'. Note that if you did not enter a study name your data will be under 'My Study'.

Gene names and identifiers for genomic data

When you loaded your genomic data we asked what type of genes, transcripts or probes you used. If you selected one of the options from the drop down menu then you can enter HUGO gene names or the identifiers in your file. If you did not select one of the options then you will need to enter the identifiers as they appear in your file.

Help! I don't see my study listed

You Local Xena Hub must be running to view any data that you have loaded into it. Please ensure it is running on your computer. You can also check which studies are on your hub and what data is in them by going to the My Computer Hub page.

Data security

How does Xena ensure the security of my data?

Xena does not utilize a central rendering service, or require hubs to be publicly accessible on the internet like, for example, the UCSC Genome Browser does. Data flows in one direction, from hubs to the user agent. If the user installs a Xena Hub on their laptop, the hub is as secure as the laptop. If the user installs a Xena Hub on a local network, behind a firewall, the hub is as secure as the local network.

The Xena Browser accesses data from a local Xena Hub on the same computer by requesting data from http://127.0.0.1. The local Xena Hub will make the data within it available at this address. The local Xena Hub will only answer requests made form the user's own computer.

Users will need to use a web browser that supports this if they wish to use a Xena Hub on the loopback interface. At the time of writing, this includes Chrome, and Firefox, but not Safari.

Is there any data that is considered to not be secure?

A very limited set of metadata is considered to be not secure in the Xena architecture model. This includes cohort names and samples names. This metadata is visible to other hubs in the following scenarios. When the user selects a cohort, all hubs are queried for samples on that cohort. When the user selects a data field, the hub holding that field is queried with the field ID (e.g. gene, probe, transcript, phenotype) and all cohort sample IDs. This means, for example, that two hubs holding data on the same cohort will see the union of sample IDs from that cohort. While data queries are not made available publicly, a malicious person could gain entry to a Xena Hub and comb through logs for these queries. For these reasons, these metadata fields should not contain private information.

To visualize and perform a KM analysis, we use two columns/rows of data, time to event and event. These data must be loaded in a phenotype file. The phenotype file can contain other data as well.

Note that you will need to name the headers in your phenotype file EXACTLY what we recognize. See the list of recognized headers for each type of survival/interval below.

This data can be in days, months, years, etc.

Time to Event and Event

Time to Event is a duration variable for each subject having a beginning and an end anywhere along the timeline of the complete study. It begins when the subject is enrolled into a study or when treatment begins, and ends when the end-point (event of interest, for example, death or metastasis) is reached or the subject is censored from the study.

Censoring means the total survival time for that subject cannot be accurately determined. This can happen when something negative for the study occurs, such as the subject drops out, is lost to follow-up, or the required data is not available or, conversely, something good happens, such as the study ends before the subject had the event of interest occur, i.e., they survived at least until the end of the study, but there is no knowledge of what happened thereafter.

Event indicates what the 'event' was for a patient, 1 for the event, for example, death or metastasis, and 0 for censored.

Help text was partially taken from A PRACTICAL GUIDE TO UNDERSTANDING KAPLAN-MEIER CURVES.

Recognized header names for different types of survival

Below is a table of the column/row header names we recognize for each type. Note that these header names are case sensitive.

Example Overall Survival

We will automatically detect and map your probes/transcripts/identifiers to HUGO gene names. For instance, we will map Affy probe IDs to HUGO gene names so that you can enter a HUGO gene name when creating a column in the Visual Spreadsheet and we will pull up the corresponding Affy probes.

Supported probes and other identifiers

• Affy U133 array (hg19) e.g. 1007_s_at

• Affy HumanExon1.0ST (hg18) e.g. 2315101

• Affy Human Gene 1.0 ST array (hg19) e.g. 7896736

• Affy Human SNP6 array (hg18) e.g. CN_473963

• Agilent Human gene expression 4X44K array (hg18) e.g. A_23_P100001

• Agilent SurePrint G3 Human CGH array 2x400K (hg18) e.g. A_16_P01651995

• Agilent Human 1A array (hg18) e.g. A_23_P149050

• Exon: GENCODE 19 e.g. ENSE00000327880.1

• Infinium HumanMethylation27 array GDC version (hg38) e.g. cg00000292

• Infinium HumanMethylation27 array TCGA legacy version (hg18) e.g. cg26211698

• Infinium HumanMethylation450 array TCGA legacy version (hg19) e.g. cg13332474

• Infinium HumanMethylation450 array GDC version (hg38) e.g. cg00000029

Supported genes and transcripts

• HUGO: human gene symbol (hg18) e.g. TP53

• HUGO: human gene symbol (hg19) e.g. TP53

• HUGO: human gene symbol (hg38) e.g. TP53

• Gene: Ensembl human genes (hg19) e.g. ENSG00000223972

• Gene: Ensembl human genes (hg38) e.g. ENSG00000223972

• Gene: GENCODE 19 e.g. ENSG00000223972.4

• Gene: GENCODE 22 comprehensive e.g. ENSG00000223972.5

• Gene: GENCODE 23 comprehensive e.g. ENSG00000223972.5

• Gene: GENCODE 23 basic e.g. ENSG00000223972.5

• Gene: UCSC Known genes (hg18) e.g. uc001aaa.1

• Gene: UCSC Known Genes (hg19) e.g. uc001aaa.1

• Transcript: GENCODE 19 comprehensive e.g. ENST00000456328.2

• Transcript: GENCODE 23 comprehensive e.g. ENST00000456328.2

• Transcript: GENCODE 23 basic e.g. ENST00000456328.2

• Transcript: RefSeq (hg19) e.g. NM_000014

• miRNA miRBase v13 stem-loop (hg18) e.g. hsa-mir-1977

• miRNA miRBase v20 stem-loop (hg19) e.g. hsa-mir-1302-2

Contact us if you don't see your gene or probe names in this list and we may be able to add it for you.

FAQ

FAQ: Xena didn't map the right probes

If it looks like we picked the wrong set of probes, please click 'Advanced' next to the 'Import' button on the last screen of the wizard to load data. You can then pick the appropriate probes.

In addition to the data itself, we require some metadata about your file. When you use our website to load your data we fill in this metadata for you. When you use the command line, you will need to provide this data in an additional file.

Metadata file requirements

The metadata file is a .json file and follows json formatting. The metadata .json file needs to be in the same directory as the data file. The metadata file and the data file need to have the same base name, including any file extensions (e.g. my_first_dataset and my_first_dataset.json OR my_second_dataset.txt and my_second_dataset.txt.json).

There are two required fields: type and cohort.

Type

Type can be:

• 'genomicMatrix' -> genomic data where samples are columns and genomic regions are rows. Note that for loading on the command line we do not support the other orientation

• 'clinicalMatrix' -> phenotypic data where samples are rows and phenotypic columns are rows. Note that for loading on the command line we do not support the other orientation

• 'mutationVector' -> mutation data

• ‘genomicSegment’-> segmented copy number data

Example:

{"type":"mutationVector"}

Cohort

Cohort is used to know if there are other data on the samples that you are loaded. You can either specify a pre-existing cohort or create your own. Cohort names are displayed on the dataset pages and the cohort drop down menu on the Heatmaps page.

For existing cohorts, you need to enter the cohort name EXACTLY as it appears as the existing cohort name. Note that our cohort names are case sensitive.

Example

{"type":"mutationVector", 
 "cohort":"TCGA Breast Cancer"}

Reference

If you are loading a mutation or segmented copy number file you will also need to specify the reference genome. You do not need to specify this for other file types

Example

{"type":"mutationVector", 
 "cohort":"TCGA Breast Cancer", 
 "assembly":"hg19"}

Probemap

If you are loading a file that has probes, transcripts, or exons and you would like to query your data by gene, you will need to provide a mapping file. You do not need to specify this for other file types.

Here is an example probemap file (a delimitated file): https://toil.xenahubs.net/download/probeMap/gencode.v23.annotation.gene.probemap

#id    gene    chrom    chromStart    chromEnd    strand 
id_1    AADACL3    chr1    12776118    12776347    +

We have many probemap files that you can see via our xenaPython app.

host =“https://reference.xenahubs.net”
xenaPython.probemap_list(host)

If you do not see a probemap that will work for you, please let us know.

To reference a probemap you need three files:

1. Include the probemap reference in your data file .json

2. Have the probemap file in the same directory as your data file and data file .json

3. Also have a .json file for the probemap so that we know how to load it

Example data file .json

{"type":"genomicMatrix", 
 "cohort":"TCGA Breast Cancer", 
 ":probeMap":"/unc_v2_exon_hg19_probe_TCGA"}

Example probemap

https://toil.xenahubs.net/download/probeMap/gencode.v23.annotation.gene.probemap

Example probemap .json file (required to be in the same folder as the probemap)

{ “type”:“probeMap”, 
  “assembly”:“hg19"}

More information about the metadata

Commands to load data

Put both your .tsv and .json files in your_home_directory/xena/files. Then run the jar, passing in the file name, like so:

java -jar cavm-0.xx.0-standalone.jar -l ~/xena/files/*

→ loads all files

OR

java -jar cavm-0.xx.0-standalone.jar -l ~/xena/files/file1.tsv

→ loads just file1.tsv

Note that you will need to substitute the name of the .jar. file As of the time of writing (September 20, 2018), the name of the .jar file was cavm-0.22.0-standalone.jar. On linux this will be in the directory where you opened the archive. On Windows or MacOS, use your operating system’s file search capability to search for cavm*jar. On Windows you will need to use the full path to your home directory, instead of “~”.

Note you do not need to load the .json files. Xena will automatically look for these and load them.

Commands to delete data

java -jar cavm-0.xx.0-standalone.jar -x ~/xena/files/file1.tsv

→ delete just file1.tsv

java -jar cavm-0.xx.0-standalone.jar -x ~/xena/files/file1.tsv ~/xena/files/file2.tsv

→ delete file1.tsv and file2.tsv

Help

You can always type:

java -jar cavm-0.xx.0-standalone.jar -h

for help.

I ran into APPLE warning you about “unidentified developer” warning when installing Xena. What do I do?

There are a couple of options. You can right-click the .dmg and chosen 'open'. You can also press the Control key, then click the app icon, then choose Open from the shortcut menu. These help pages might help: http://www.iclarified.com:8081/28180/how-to-open-applications-from-unidentified-developers-in-mac-os-x-mountain-lion and from Apple: https://support.apple.com/kb/PH25088?locale=en_US .

I see my probes/genes/transcripts when loading my data, but I don't know whether to choose hg18, hg19 or hg38?

The only time the assembly matters is if you decided to visualize part or all of a chromosome, rather than a gene/probe/transcript. If you want to visualize only genes/probes/transcripts than it does not matter which assembly you choose.

I don't see my study listed in the visualization

You Local Xena Hub must be running to view any data that you have loaded into it. Please ensure it is started up. You can also check which studies are on your hub and what data is in them by going to the My Computer Hub page: xenabrowser.net/datapages/?host=https%3A%2F%2Flocal.xena.ucsc.edu%3A7223.

You also may not see your study if the hub is still loading the data. Wait a few minutes and refresh the page.

I'm entering a gene name but it only draws gray

When you loaded your genomic data we asked what type of genes, transcripts or probes you used. If you selected one of the options from the drop down menu then you can enter HUGO gene names or the identifiers in your file. If you did not select one of the options then you will need to enter the identifiers as they appear in your file.

Can I load two or more phenotype files to the same study?

Yes, we will allow you to select phenotypes from both files in the visualization.

Help! My file is too large to open in Microsoft Excel.

You might be able to load your file anyways, depending on the format. Give it a try and if you are unable to load it, write us an email and we may be able to fix your file for you.

How do I convert my .xls or .xlsx into a tab-delimited file?

You can export a Microsoft Excel file as a tab-delimited file using the 'Save as ...' function.

Unix vs DOS

We require that your data files have a unix line ending. To ensure that your files have this line ending on a DOS, please follow the help here:

Note that this requirement is only for data files, not for the associated .json files.

There are 2 basic data formats and 2 advanced data formats. Each of these formats has one or more biological data types that it supports.

General Specifications for all data formats

We support most types of genomic and phenotype/clinical/sample annotations. For genomic data we support calls made on the raw data including but not limited to expression calls, mutation calls, etc. This is what TCGA calls ‘Level 3’ data and is typically a value on gene, transcript, probe, etc. We do not support FASTQ, BAMs, or other ‘raw’ files. Please contact us if you have any questions.

We support tab-delimited and Microsoft Excel files (.xlsx and .xls). Tab-delimited files generally have a file name ending in .tsv or .txt, though we do not require this. Note that we load tab-delimited files much faster than Excel files. You can export a Microsoft Excel file as a tab-delimited file using the 'Save as ...' function.

Please do not have any duplicate genes/probes/identifiers or samples. We will allow you to load with duplicates but will only display the first one encountered in the file.

We assume you use a '.' to indicate a decimal place as opposed to a ',' .

Here is a with example data in addition to the examples below.

Basic Genomic data: numbers in a rectangle/matrix/spreadsheet

These are numeric data called on genomic regions (e.g. exon expression or gene-level copy number). This data is in a rectangle where samples are columns and rows are the genomic regions (e.g. HUGO gene symbol, transcript ID, probe ID, etc). We also support samples as rows and genomic regions as the columns (i.e. the opposite orientation). For supported genomic regions, please see .

Supported data types

• RNA-seq expression (exon, transcript, gene, etc)

• Array-based expression (probe, gene, etc)

• Gene-level mutation

• Gene-level copy number

• DNA methylation

• RPPA

• and more ...

An example of a genomic matrix file (in this case, expression):

Basic Phenotypic data: categories or non-genomic in a rectangle/matrix/spreadsheet

These are data on a sample or patient that is categorical in nature (e.g. Tumor Stage or 'wild type' or 'mutant' for a gene) or is numerical but non-genomic (e.g. age or a genomic signature). Samples can be columns and rows can be phenotype/clinical/sample orientation or vice versa. We support both orientations.

Supported data types

• phenotype/patient/clinical data (age, weight, if there was blood drawn, etc)

• sample/aliquot data (where it was sequenced, tumor weight, etc)

• derived data (regulon activity for a gene, etc)

• genomic signatures (EMT signature score, stemness score, etc)

• other (whether a sample has an ERG-TMPRSS2 fusion, whether a sample has WGS data available, etc)

Categorial vs numerical data

We support both numerical and categorical data. For numerical data please use a blank field for any samples which may be missing data. For categorical data you can use a blank field or "NA" for any samples which may be missing data.

To have it be treated as a numerical field please use a blank field.

An example of a phenotype matrix file:

Advanced Segmented data

For segmented data, we require the following 5 columns: sample, chr, start, end, and value. Note that your column headers must be these names exactly!

Please use 'NA' to indicate no data.

Supported data types

• copy number

We currently accept hg38, hg19, hg18 coordinates.

Example segmented copy number data with required columns:

Advanced Positional data

For positional data, we require 6 columns: sample, chr, start, end, reference, alt. Note that your column headers must be these names exactly!

Supported data types

• mutation data

We currently accept hg38, hg19, hg18 coordinates.

Example mutation data with the six required columns, plus the gene column:

Advanced Other data

Institutional Xena Hubs allow you to share data, visualizations, and analyses with a specific group of people. Xena Hubs can be set up on any server or in the cloud. You control who has access to the Xena Hub by controlling who has access to the server on which it is hosted.

To make your data publicly available, simply make the server open to the web.

Download

First, download the ucsc_xena_xxx.tar.gz file to your server, here:

The file to download is the one called "Tar archive, no updater or JRE - recommended for linux server developments". Uncompress and extract the .jar file (cavm-xxx-standalone.jar). The current version is 0.25.0.

Start the hub

The hub can be started with "java -jar cavm-xxx-standalone.jar". Passing option --help will display usage information.

Note that you need to use Java 8 to run the hub.

There are several options you will want to set.

To bind an external interface (instead of loopback), use "--host 0.0.0.0".

The connection between your hub and the Xena Browser is through https, use "--certfile" and "--keyfile" options to set them.

There are three paths that can be configured: the database file, the log file, and the root directory for data files to be served. These are set by --database, --logfile, and --root. If you don't set these, they will default to paths under ${HOME}/xena.

Example start script for an open-access hub

Copy the content below to a file "start_script"

Link server.jar to cavm-x.xx.x-standalone.jar

Make "start_script" executable

Run "./start_script"

Your hub is now running on "https://computer-external-ip:7223".

Getting a security certificate for an open-access hub

When a Xena Hub starts, it opens two consecutive ports, for http and https connections, e.g. 7222 and 7223. HTTP is always the lower number, and HTTPS is always the higher number. This means your hub has two urls

Connecting via HTTP to the hub is no longer supported by modern web browsers, thus you will need to connect via HTTPS. To do this you will need an HTTPS certificate and private key. Paths to the cert and key are set with --certfile and --keyfile. This might seem redundant for a hub behind a firewall, but the web app has no influence over the security policies of the web browser. HTTPS certificates can be acquired from free public Certificate Authorities, or via NIH InCommon.

Make your data ready

Load data through command line

Once the hub is running, and input files have been placed in the --root directory, a file can be loaded by running the jar a second time, with the -l option, like

Delete data through command line

If your hub is run on the default 7222 port, you can load data with

If your hub is running on a different port, you load data with

Please contact us at genome-cancer@soe.ucsc.edu for more assistance.

If your hub is run on the default 7222 port, you can delete data with

If your hub is running on a different port, you delete data with

Viewing data from the hub

You can now go to the visualization and add a cohort or study listed in your hub.

If you don't have a security certificate yet

If you don't have a security certificate yet but you would like to verify that the hub is working you can use ssh tunneling. An example of how to do this for AWS is below, where it is assumed that the xena hub is running on port 7222 for http and 7223 for https. In this scenario, you start the hub without using --certfile and --keyfile options.

Assuming that you typically ssh into EC2 on AWS like this,

you will now set up an ssh tunnel to port 8000 on your computer. To do this we add the -L option:

Now on your computer, http://localhost:8000 is the same as the http://aws-ip:7222. Chrome Browser does not allow a connection to http://aws-ip:7222, but it will allow a connection to http://localhost:8000.

An example apache configuration on AWS VM

in /etc/httpd/conf/httpd.conf

A landing page for my hub

How do I add a 'Launch Xena' button like the TOIL landing page

<button class="hubButton" data-cohort="TCGA TARGET GTEx">Launch Xena</button>

To add a clickable button in the hub landing page, make sure the button has classname 'hubButton'. You also need to specify the cohort to view, defined by the data parameter 'data-cohort'. Once users click the button, the visualization wizard will be launched to the specified cohort. You can change the button label.

A landing page for my cohort

How do I add a "Launch" button like the TCGA TARGET GTEx landing page

<button class="cohortButton" data-bookmark="bc7f3f46b042bcf5c099439c2816ff01">Example: compare FOXM1 expression</button>

The button must has a classname 'cohortButton'. If you have the data parameter 'data-bookmark', clicking the button will take the user to the bookmark view. If you don't have the 'data-bookmark' parameter, clicking the button will take the user to the visualization wizard with an empty spreadsheet. You can change the button label. You can as many button as you want.