今天将分享minGPT的pytorch安装与使用的完整过程，为了方便大家学习理解整个流程，将整个流程步骤进行了整理，并给出详细的步骤结果。感兴趣的朋友赶紧动手试一试吧。
一、minGPT安装
openai公司开发的GPT模型有上亿参数，模型非常大，可以参考github项目https://github.com/openai/gpt-2。为了能在常规的电脑上可以运行GPT模型，将安装minGPT模型。
安装minGPT的依赖库，在命令行使用以下命令安装，安装完成后可以看到成功安装所需的依赖库。
代码语言：javascript复制git clone https://github.com/junqiangchen/minGPT.git
cd minGPT
pip install -e .
二、minGPT使用案例
这里不讲述如何训练minGPT，具体的训练步骤可以参考这个例子：https://github.com/junqiangchen/minGPT/blob/master/demo.ipynb
这里直接拿gpt预训练权重来生成测试，预训练权重有四个可以选择：'gpt2', 'gpt2-medium', 'gpt2-large', 'gpt2-xl'，其中gpt2模型参数有124.44M，gpt2-medium模型参数有354.82M，gpt2-large模型参数有774.03M，gpt2-xl模型参数有1557.61M。如下是gpt2-medium预训练权重下载过程。
import torch
from transformers import GPT2Tokenizer, GPT2LMHeadModel
from mingpt.model import GPT
from mingpt.utils import set_seed
from mingpt.bpe import BPETokenizer
set_seed(3407)
use_mingpt = True # use minGPT or huggingface/transformers model?
model_type = 'gpt2-medium'
device = 'cuda'
if use_mingpt:
model = GPT.from_pretrained(model_type)
else:
model = GPT2LMHeadModel.from_pretrained(model_type)
model.config.pad_token_id = model.config.eos_token_id # suppress a warning
# ship model to device and set to eval mode
model.to(device)
model.eval()
def generate(prompt='', num_samples=10, steps=200, do_sample=True):
# tokenize the input prompt into integer input sequence
if use_mingpt:
tokenizer = BPETokenizer()
if prompt == '':
# to create unconditional samples...
# manually create a tensor with only the special <|endoftext|> token
# similar to what openai's code does here
# https://github.com/openai/gpt-2/blob/master/src/generate_unconditional_samples.py
x = torch.tensor([[tokenizer.encoder.encoder['<|endoftext|>']]], dtype=torch.long)
else:
x = tokenizer(prompt).to(device)
else:
tokenizer = GPT2Tokenizer.from_pretrained(model_type)
if prompt == '':
# to create unconditional samples...
# huggingface/transformers tokenizer special cases these strings
prompt = '<|endoftext|>'
encoded_input = tokenizer(prompt, return_tensors='pt').to(device)
x = encoded_input['input_ids']
# we'll process all desired num_samples in a batch, so expand out the batch dim
x = x.expand(num_samples, -1)
# forward the model `steps` times to get samples, in a batch
y = model.generate(x, max_new_tokens=steps, do_sample=do_sample, top_k=40)
for i in range(num_samples):
out = tokenizer.decode(y[i].cpu().squeeze())
print('-' * 80)
print(out)
if __name__ == '__main__':
generate(prompt='sam altman,the', )
如上所示输入“sam altman,the”，给出来10条结果，给出来的结果还是五花八门的，挺有意思的。
代码语言：javascript复制--------------------------------------------------------------------------------
sam altman,the_masta_smith,andy_palin (8:48:23 PM): im gonna get some drinks
ghostsrrelz (8:48:48 PM): whats your username?
sweet_erin78 (8:48:52 PM): xxx
ghostsrrelz (8:49:09 PM): im like 12
8:48 PM (s2l) is dead
sasquatch0010 (8:49:14 PM): who is it?
sweet_erin78 (8:49:24 PM): :3
sasquatch0010 (8:49:40 PM): idk im pretty cool
sweet_erin78 (8:49:52 PM): :3
8:48 PM (s2l) has joined the chat.
8:48 PM (2b2t) has joined the chat.
8:48
--------------------------------------------------------------------------------
sam altman,the man who did the hard manning
"When the time will come when our men should no longer be necessary in battle, I will begin to write the words I have promised them
to be my son, and my spirit will roam over them with wild wings, and with joy of glory
of soul and body."
-Joseph Smith, Journal of Discourses, #20
Joseph Smith, the man who did the hard manning, who helped lay the foundation of the Church, was born in Illinois on July 23, 1843. From the birth of his mother, Mrs. Joseph Young was taught that "our fathers laid the foundations of the kingdom." In fact, according to the teachings of Joseph Smith, their name is "The Smith Family":
"We are of the same family; and all were of the seed of Abraham; and we were created for the purpose of setting the man upon the earth, in order to execute the work of the Lord
--------------------------------------------------------------------------------
sam altman,the world is a better place then it is today; you are welcome at the top.the only thing missing is me.
Anonymous 09/08/17 (Fri) 14:50:36 AM No. 232353 >>232354
I've already talked about the whole lack of social anxiety (which isn't really a thing, it's like how there's always been anxiety and fear about being alone with yourself) but this is such a poor idea.
What does this actually do to someone? In any case, she's definitely a bad motherfucker. She should be arrested for child abuse. I've already talked about the whole lack of social anxiety (which isn't really a thing, it's like how there's always been anxiety and fear about being alone with yourself) but this is such a poor idea.What does this actually do to someone? In any case, she's definitely a bad motherfucker. She should be arrested for child abuse.
--------------------------------------------------------------------------------
sam altman,the band in question,will join the ranks of the deceased here as well as an incredible lineup that includes members from the band's previous incarnation,the legendary psychedelic punk quintet,The Velvets. Their new material showcases their best work in more subtle ways while not completely eliminating any of these elements that helped define the past three-plus decades of punk rock. "There's so much to love here," says bassist/pianist/guitarist Alex Jones. "This album might be their most diverse yet. There's a ton of different sounds and emotions mixed in with some of their classic songs like"I'm the One,"("A Million Voices"),and"Ain't That Love" (not sure why I kept listening to this one). There's a lot of great live energy right now and they still have that punk rock energy that is on point, but they also show a new direction. I like how they have that attitude of, 'hey, there is nothing wrong
--------------------------------------------------------------------------------
sam altman,the daniel dreier quote,the santoku quote,santa bellino,so many others. I guess that would be my top ten. Anytime I write something I post it HERE, so stay tuned for updates. Here a few more links; http://www.danieldreier.com/how-do-you-work-harder-for-cheap-money/ http://www.zdnet.com/the-how-to-work-harder-for-cheap-money-a24753926 http://www.nytimes.com/2007/09/19/top-10-less-compelling-debunkers-of-cheap-money.html?pagewanted=all http://www.money.co.nz/how-to-know-how-much-money-you-need-to-make.html http://www.money.co.nz/
--------------------------------------------------------------------------------
sam altman,the "S.A." team)
It's about my life at this point:
-A lot of the time I'm with my mom, I'm a little nervous, I don't really know what to do, I get frustrated and I try to hide that.
-I have a lot of things I miss, like going outside, the sun doesn't always seem to be so bright here sometimes, the music is pretty quiet sometimes.
-The only time I go anywhere is the day I went to high school.
-I was raised in New Jersey, my sister's family moved here recently, and we moved here after our parents left us. My mom still works at the DMV, and she knows a lot of the guys here.
-I guess I'm in between, not really sure what to do, but I'm trying to be patient, because this is just a big city.
-It's tough in some ways because
--------------------------------------------------------------------------------
sam altman,the author of the book 'How to get rich in the bitcoin economy?'
I recently found myself being approached by a bitcoin trader. He was asking the price of bitcoin to be set at 1000. While my response was that its value cannot be increased by 1,000 since the price is already 1000. I was not going to give bitcoin that advice. He said if I said that in any way I'll lose my money. He was not interested in learning anything.
I am not interested in his idea though.
He simply wanted to know how much Bitcoin he could get. I can only tell him that because there are no exchanges to trade bitcoins with. I had no idea what he was talking about in that respect and was curious as to the reasoning behind it.
I found our conversation entertaining. He mentioned he was a big fan of the concept and wanted to know if the idea could work here in my market. I said sure sure I'll keep my word to do
--------------------------------------------------------------------------------
sam altman,the most recent episode was all about, well, making art again and making a good show, but you should watch it before you get any more into it, because you are going to find that it is great.The show was filmed and mixed in LA on a $500,000 budget, so even with just your standard crappy studio shots and effects, it is the kind of show that you will want to go see if you can and maybe do a bunch of other things with it. You are not in Los Angeles. You are in Scotland. I was in Glasgow so... and here's the kicker. The show is really good. I am not sure you'll agree, but I think it is one of the better-made horror series on television.I thought the story of that episode was brilliant and it was a good way to introduce the idea of "we're a group of guys who play these games, we have a game for them and we're trying to stop evil from going all over
--------------------------------------------------------------------------------
sam altman,the artist,the artist,was a brilliant artist.she was a master.she was a great artist.he is indeed amazing,his work has been so many times published in several newspapers and journals such places as,the journal-magazine , the magazine ,... she is the best artist ever.i love this artist,she is really amazing
What is the secret of his work?
There are several secrets of a person,the best way is with the art,the art is the essence,the art is the spirit.
The secret is his talent,his talent,the art is so rare,even if it is a simple painting,it's still a masterpiece.
So the secret can be divided into two different types of secret:
· The secret "can only be seen."he cannot tell a soul.
·The secret "must be done to be seen."he must do everything to be viewed.
·The secret "has
--------------------------------------------------------------------------------
sam altman,the_great_sweden,1,110618,btc,11/14/17 11:06,,www.reddit.com/r/btc/comments/7cbxwc,11/14/2017 5:03,0,Is everyone's bitcoin wallet hacked at the same time? 10731916,0,Bitcoin price crash on MtGox,11/14/17 11:06,,www.reddit.com/r/btc/comments/7cbxwc,11/14/2017 5:02,1,[Bitcoin cash] People are selling up on Bitcoin cash for good 10731917,0,Bitcoin mining speed surge on Bitmain,11/14/17 11:07,,www.reddit.com/r/btc/comments/7cbxwc,11/14/2017 5:02,0,What can you do to help Bitcoin. This is getting out of control. The
