从CNCB下载单细胞转录组fastq文件并定量

2023-03-18 14:58:30 浏览数 (2)

安装CellRanger

https://support.10xgenomics.com/single-cell-gene-expression/software/downloads/latest?

代码语言:shell复制
cd ~/APP
wget -O cellranger-7.1.0.tar.xz "https://cf.10xgenomics.com/releases/cell-exp/cellranger-7.1.0.tar.xz?Expires=1674686023&Policy=eyJTdGF0ZW1lbnQiOlt7IlJlc291cmNlIjoiaHR0cHM6Ly9jZi4xMHhnZW5vbWljcy5jb20vcmVsZWFzZXMvY2VsbC1leHAvY2VsbHJhbmdlci03LjEuMC50YXIueHoiLCJDb25kaXRpb24iOnsiRGF0ZUxlc3NUaGFuIjp7IkFXUzpFcG9jaFRpbWUiOjE2NzQ2ODYwMjN9fX1dfQ__&Signature=kBEWDvHNogaTb-bmHzd7kHIBIfs8LQmePXusMXNKYpuqbgNHX5Ai0mhB-wv6ig1X5QFZytrl4gYXd8DyNXVo85hRp6Iw9k1UHtuuictpnyNe-5JNocePNKou89k9AOEGoatr6xa9z1VPkOj33FTpG25OpfQowpBrwvNhFF27qoSqw7EcjSWm53zB4QsYqMR~Bi-5MXTEplAxusXnE5A1HtVOo31lsL7cavd88ez9yFcSDIf65~KJR6KqDzqYS3NAcm3MKBWWSeIVAwOWAuHaQONeAew8X4fMb3ql85CpeaCWrQdB-vlUVkQbM0gJY2S7MQ9SJ0B5qUc7qo9UWLXATw__&Key-Pair-Id=APKAI7S6A5RYOXBWRPDA"
tar -xf cellranger-7.1.0.tar.xz
ln -s ~/APP/cellranger-7.1.0/bin/cellranger ~/APP/bin/cellranger

export PATH=$HOME/APP/bin:$PATH

index

10X提供人和鼠的基因组参考index,其他物种可以是用cellranger自行构建

代码语言:shell复制
# Human reference (GRCh38)  md5sum: dfd654de39bff23917471e7fcc7a00cd
wget https://cf.10xgenomics.com/supp/cell-exp/refdata-gex-GRCh38-2020-A.tar.gz
md5sum refdata-gex-GRCh38-2020-A.tar.gz
tar -xf refdata-gex-GRCh38-2020-A.tar.gz

下载原始数据

CNCB的原始数据需要申请,申请通过后可以下载meta data

https://ngdc.cncb.ac.cn/gsa-human/browse/HRA000728

run信息 run.tab文件

代码语言:text复制
Accession       Run title       Experiment accession    Run data file type      File name 1     MD5 checksum 1  DownLoad1       File name 2     MD5 checksum 2
HRR208788       Y35_L.R1        HRX170150       fastq   HRR208788_f1.fastq.gz   cadeb85c01d10660830651c58f320dc3                HRR208788_r2.fastq.gz   e1bc997bf7ae94806636415c7a552fff
HRR208789       Y35_L.R2        HRX170150       fastq   HRR208789_f1.fastq.gz   07ae41fb8e962dacb112d8d6d6bdb940                HRR208789_r2.fastq.gz   63ed718855993f248dfe37557e17586e
HRR208790       Y35_L.R3        HRX170150       fastq   HRR208790_f1.fastq.gz   f787b6ee5a94fd054cf0352f00b5b7bd                HRR208790_r2.fastq.gz   ed156f85be6247f27185db9c38170f0c
HRR208791       Y35_L.R4        HRX170150       fastq   HRR208791_f1.fastq.gz   5db5a61729a1c7fb332cd002540c1569                HRR208791_r2.fastq.gz   e21cfcfca9e1722b148f2a26c586f624
HRR208792       Y35_R.R1        HRX170151       fastq   HRR208792_f1.fastq.gz   3886b0cf3a68eeff8224bee725b9b761                HRR208792_r2.fastq.gz   2d3047a809a2888bc5df47b2a1e57055
HRR208793       Y35_R.R2        HRX170151       fastq   HRR208793_f1.fastq.gz   37c08ac3019aabc6158dbf5a39bc3fc4                HRR208793_r2.fastq.gz   8714f1b0a5f81f6126555e5e012273a0
HRR208794       Y35_R.R3        HRX170151       fastq   HRR208794_f1.fastq.gz   4f186e72b0c95ae477f3ab17c377acac                HRR208794_r2.fastq.gz   efa15e66fec4b1246774823b942b2f6e
HRR208795       Y35_R.R4        HRX170151       fastq   HRR208795_f1.fastq.gz   47db73fb1507e8ad034f9b5fd96b1112                HRR208795_r2.fastq.gz   c33d9a77063aeefa227b2f9f89b3ad05
HRR208796       Y44_L.R1        HRX170152       fastq   HRR208796_f1.fastq.gz   1ea8e5f0d87c24d69045b7f61d0a36d7                HRR208796_r2.fastq.gz   8dbdddfd49bc99b4cf72c2ac5da1c455
  • 制作下载文件
代码语言:shell复制
sed '1d' run.tab | cut -f 1 >> hrr
sed '1d' run.tab | cut -f 1 >> hrr
sed '1d' run.tab | cut -f 5 >> fq
sed '1d' run.tab | cut -f 8 >> fq
paste -d '/' hrr fq > files
rm -f hrr fq
  • 下载
代码语言:shell复制
#>>>down.sh>>>
HRA=HRA000728
cat files | while read i
do
echo ${i}
wget -c --user 注册账户的邮箱 --password 注册账户的密码  --mirror ftp://human.big.ac.cn/${HRA}/${i}
done
#<<<down.sh<<<
代码语言:shell复制
nohup bash down.sh &> down.sh.log &
  • 把fastq文件移动到同一个目录下
代码语言:shell复制
find . -type f -name "*.fastq.gz" > fq

cat fq | while read i
do
mv ${i} fqs
done

rm -rf human*
rm -f fq

ln -s ~/Project/HM/fqs/* ~/Project/HM/fastqs/

# 56个文件
find fqs -name *gz | wc -l 

重命名脚本

10X官网给指出来了文件名规则:

https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/fastq-input

https://support.illumina.com/help/BaseSpace_OLH_009008/Content/Source/Informatics/BS/NamingConvention_FASTQ-files-swBS.htm

Sample Name_S1_L00Lane Number_001.fastq.gz

Where Read Type is one of:

  • I1: Sample index read (optional)
  • I2: Sample index read (optional)
  • R1: Read 1
  • R2: Read 2

用python制作重命名脚本

代码语言:shell复制
from collections import Counter
import numpy as np
import pandas as pd
import bioquest as bq

df=pd.read_csv('run.tab',sep='t')
# Sample Name
sn=np.repeat(df.loc[:,"Run title"],2)
# Read Type
r= [1,2] * df.shape[0]
sn=[x.split('.')[0] for x in sn]
sn = bq.st.removes(string=sn,pattern=r"_")
# Sample Number; Lane Number
s,l = [],[]
for x,y in enumerate(Counter(sn).values()):
    l  = list(range(1,int(y/2) 1))
    s  = [x 1] * y
l=np.repeat(l,2)
# new file name
new=[f'{a}_S{b}_L00{c}_R{d}_001.fastq.gz' for a,b,c,d in zip(sn,s,l,r)]

# old file name
a=[[x,y] for x,y in  zip(df.loc[:,"File name 1"],df.loc[:,"File name 2"])]
old=np.array(a).flatten()

#shell script
df2=pd.DataFrame({1:old,2:new})
df2.insert(0,column=0,value='mv')
df2.to_csv('rename.sh',index=False,header=False,sep=' ')

rename.sh前几行

代码语言:shell复制
cd fastqs
bash rename.sh
代码语言:shell复制
mv HRR208788_f1.fastq.gz Y35L_S1_L001_R1_001.fastq.gz
mv HRR208788_r2.fastq.gz Y35L_S1_L001_R2_001.fastq.gz
mv HRR208789_f1.fastq.gz Y35L_S1_L002_R1_001.fastq.gz
mv HRR208789_r2.fastq.gz Y35L_S1_L002_R2_001.fastq.gz
mv HRR208790_f1.fastq.gz Y35L_S1_L003_R1_001.fastq.gz
mv HRR208790_r2.fastq.gz Y35L_S1_L003_R2_001.fastq.gz
mv HRR208791_f1.fastq.gz Y35L_S1_L004_R1_001.fastq.gz
mv HRR208791_r2.fastq.gz Y35L_S1_L004_R2_001.fastq.gz

cellranger count

代码语言:shell复制
#>>>quantify.sh
fastqs_dir=~/Project/HM/fastqs
index_dir=~/DataHub/Genomics/10X/refdata-gex-GRCh38-2020-A
output_dir=~/Project/HM/quantify
cd ${output_dir}

ls ${fastqs_dir} | cut -d '_' -f 1 | uniq | while read i
do
cellranger count 
    --id $i 
    --transcriptome ${index_dir} 
    --fastqs ${fastqs_dir} 
    --sample $i 
    --localcores 12 
    --localmem 128
done
#<<<quantify.sh

nohup bash quantify.sh &> quantify.sh.log &

输出文件

可以参考生信技能树https://mp.weixin.qq.com/s/VWUmJZnzT7m_7QDjxkbrJw

需要查看的文件

  • web_summary.html:这个是必须要看的,粗略浏览本次10x样本走cellranger count流程的运行质量
  • filtered_feature_bc_matrix.h5: Python读取表达量矩阵

已经得到表达量矩阵下一步走scanpy分析流程。

Reference

代码语言:shell复制
https://mp.weixin.qq.com/s/3njDJlQjunWfGfdh-iMZNA
https://www.plob.org/article/20961.html
https://www.jianshu.com/p/744371e03894

0 人点赞