免费AI视频生成工具:硅基流动SiliconCloud上的CogVideoX-2B模型

2024-08-29 18:50:21 浏览数 (2)

CogVideoX-2B 是由智谱AI开发的一款开源视频生成模型,其核心在于3D变分自编码器(3D VAE)技术,这项技术能够将视频数据压缩至原来的2%,极大地降低了模型处理视频时所需的计算资源。此外,为了进一步提升内容的连贯性,CogVideoX-2B采用了3D旋转位置编码(3D RoPE)技术,使得模型在处理视频时能够更好地捕捉时间维度上的帧间关系,建立起视频中的长期依赖关系,从而生成更加流畅和连贯的视频序列。

该模型在FP-16精度下的推理需18GB显存。其中所有模型和配置文件一共9GB 大小。CogVideoX-2B的提示词上限为226个token,视频长度为6秒,帧率为8帧/秒,视频分辨率为720*480。

网址:https://github.com/THUDM/CogVideo

此外,CogVideoX-2B支持在微调后用于微短剧等视频内容的基本制作,这一开源模型仅需个人消费级显卡就能进行推理运行,极大降低了模型使用门槛,能够让更多开发者、初创团队、企业开发适合自己使用的视频生成模型。开发者们可以在提示词优化、视频长度、帧率、分辨率、场景微调以及围绕视频的各类功能开发上贡献开源力量。

不过,如果在本地部署CogVideo,不仅需要较高的硬件配置,而且部署过程比较繁琐,非专业人士需要花较长的时间和精力才能部署成功。

好在硅基流动SiliconCloud上已经部署好现成的CogVideoX-2B模型,我们直接开箱即用,能联网就行,非常简单易用。

体验地址是:https://cloud.siliconflow.cn/playground/text-to-video

右边是一些设置:

Model是CogVideoX-2B,目前仅有这一个,以后估计还会新添加其他的AI视频生成模型。

Width是视频的宽,默认720,不可调整;

Height是视频的高,默认480,不可调整;

Seed是种子,相同的种子和提示词可以产生相似的视频。

最底部是提示词输入框,已经有几个默认的提示词样本,比如:

提示词1:A street artist, clad in a worn-out denim jacket and a colorful bandana, stands before a vast concrete wall in the heart, holding a can of spray paint, spray-painting a colorful bird on a mottled wall.(一位街头艺术家,身穿一件破旧的牛仔夹克和一条色彩斑斓的头巾,在市中心的一堵巨大的混凝土墙面前,手持一罐喷漆,正在斑驳的墙上喷涂一只色彩斑斓的鸟。)

提示词2:A detailed wooden toy ship with intricately carved masts and sails is seen gliding smoothly over a plush, blue carpet that mimics the waves of the sea. The ship's hull is painted a rich brown, with tiny windows. The carpet, soft and textured, provides a perfect backdrop, resembling an oceanic expanse. Surrounding the ship are various other toys and children's items, hinting at a playful environment. The scene captures the innocence and imagination of childhood, with the toy ship's journey symbolizing endless adventures in a whimsical, indoor setting.(一艘精致的木制玩具船,船桅和帆布上雕刻着复杂的图案,被看到在一块模仿海浪的柔软蓝色地毯上平稳地滑行。船体被漆成浓郁的棕色,并有小小的窗户。地毯柔软且质感丰富,提供了一个完美的背景,看起来就像一片海洋。玩具船周围是各种其他玩具和儿童用品,暗示着一个充满乐趣的环境。这个场景捕捉了童年的纯真和想象力,玩具船的旅程象征着在一个奇妙的室内环境中的无尽冒险。)

提示词3:In the haunting backdrop of a war-torn city, where ruins and crumbled walls tell a story of devastation, a poignant close-up frames a young girl. Her face is smudged with ash, a silent testament to the chaos around her. Her eyes glistening with a mix of sorrow and resilience, capturing the raw emotion of a world that has lost its innocence to the ravages of conflict.(在一座饱受战争摧残的城市的凄凉背景下,废墟和破碎的墙壁诉说着毁灭的故事,一个感人的特写镜头捕捉了一个小女孩的形象。她的脸上沾满了灰烬,默默证明了她周围混乱的场面。她的眼睛闪烁着悲伤和坚韧的混合情感,捕捉了一个因冲突的蹂躏而失去纯真的世界的原始情感。)

提示词4:The camera follows behind a white vintage SUV with a black roof rack as it speeds up a steep dirt road surrounded by pine trees on a steep mountain slope, dust kicks up from its tires, the sunlight shines on the SUV as it speeds along the dirt road, casting a warm glow over the scene. The dirt road curves gently into the distance, with no other cars or vehicles in sight. The trees on either side of the road are redwoods, with patches of greenery scattered throughout. The car is seen from the rear following the curve with ease, making it seem as if it is on a rugged drive through the rugged terrain. The dirt road itself is surrounded by steep hills and mountains, with a clear blue sky above with wispy clouds.(摄像机跟随在一辆白色复古SUV后面,车顶装有黑色行李架,它在陡峭的土路上加速行驶,周围是松树环绕的陡峭山坡,轮胎扬起尘土。阳光照射在SUV上,随着它在土路上飞驰,给场景带来了温暖的光芒。土路缓缓地弯曲延伸至远方,视线中没有其他车辆。道路两旁是红杉树,周围散布着片片绿色植被。汽车从后方看去,轻松地沿着曲线行驶,给人一种它正在崎岖地形中进行一次粗犷驾驶的感觉。土路本身被陡峭的山丘和山脉所环绕,上方是晴朗的蓝天,飘着几缕薄云。)

看起来,提示词需要比较详尽而具体,可以让硅基流动SiliconCloud上的文本生成模型来帮忙。

点击左侧的文本对话,输入提示词:

写一个AI生成视频的提示词(用英文输出),主题是“可爱的小女孩在吹蒲公英”,提示词可以借鉴参考下面的范例:

提示词1:A street artist, clad in a worn-out denim jacket and a colorful bandana, stands before a vast concrete wall in the heart, holding a can of spray paint, spray-painting a colorful bird on a mottled wall.

提示词2:A detailed wooden toy ship with intricately carved masts and sails is seen gliding smoothly over a plush, blue carpet that mimics the waves of the sea. The ship's hull is painted a rich brown, with tiny windows. The carpet, soft and textured, provides a perfect backdrop, resembling an oceanic expanse. Surrounding the ship are various other toys and children's items, hinting at a playful environment. The scene captures the innocence and imagination of childhood, with the toy ship's journey symbolizing endless adventures in a whimsical, indoor setting.

提示词3:In the haunting backdrop of a war-torn city, where ruins and crumbled walls tell a story of devastation, a poignant close-up frames a young girl. Her face is smudged with ash, a silent testament to the chaos around her. Her eyes glistening with a mix of sorrow and resilience, capturing the raw emotion of a world that has lost its innocence to the ravages of conflict.

提示词4:The camera follows behind a white vintage SUV with a black roof rack as it speeds up a steep dirt road surrounded by pine trees on a steep mountain slope, dust kicks up from its tires, the sunlight shines on the SUV as it speeds along the dirt road, casting a warm glow over the scene. The dirt road curves gently into the distance, with no other cars or vehicles in sight. The trees on either side of the road are redwoods, with patches of greenery scattered throughout. The car is seen from the rear following the curve with ease, making it seem as if it is on a rugged drive through the rugged terrain. The dirt road itself is surrounded by steep hills and mountains, with a clear blue sky above with wispy clouds.

提示词非常快的生成完成了:

In a tranquil summer afternoon, a child filled with innocence stands amidst a golden sea of dandelions. The gentle breeze caresses the flowers, making them sway like fairy dancers in nature. The child, dressed in a pink dress, wears a small straw hat on her head, holding a slender dandelion stem in her hand. With a gentle breath, she blows, sending the dandelion seeds floating like snowflakes, spinning in the air before gently settling, like tiny stars, into the sea of flowers. Surrounding her is lush green grass, with rolling hills in the distance. Sunlight, filtering through the clouds, casts a warm glow, enhancing the dreamy and heartwarming atmosphere of the scene. The child's smile is like the warmth of summer sun, warm and pure, her eyes brimming with curiosity and love for the world, as if telling us that every fleeting moment is a beautiful time of dance between nature and innocence.(在一片宁静的夏日午后,一个充满童真的小女孩站在一片金黄的蒲公英花海中,微风轻拂,蒲公英随风轻轻飘动,仿佛是大自然的精灵在舞蹈。小女孩身穿一件粉色的连衣裙,头上戴着一顶小巧的草帽,手中拿着一根细长的蒲公英茎,她轻轻一吹,蒲公英种子如雪花般飞舞,轻盈地在空中旋转,随后缓缓降落,仿佛是天空中的小星星,散落在花海之中。周围是绿意盎然的草地,远处是连绵起伏的山丘,阳光透过云层,洒下温暖的光芒,为这个场景增添了几分梦幻与温馨。小女孩的笑容如同夏日的阳光,温暖而纯净,她的眼神中充满了对世界的好奇与爱,仿佛在告诉我们,每一个小小的瞬间,都是大自然与童真共舞的美好时光。)

输入提示词之后,会显示提示:视频生成中,将在几分钟内完成。

相对于图片生成,视频生成需要耗费大量算力,因而需要较长时间,大概几分钟,需耐心等待。

大概等待了2分钟后,一段6秒的视频生成完成:

整体上比较符合提示词的要求,但是小女孩的嘴巴有些异常,导致看起来不是可爱,而是有些恐怖。看来,这个CogVideoX-2B视频生成模型还比较粗糙,只能作为个人体验用,还没法生成高质量的视频,无法商用。

在视频上点击右键,选择视频另存为,就可以将生成的视频保存下来了。

0 人点赞