Spring学习笔记（三十六）——SpringBoot 实现大文件分片上传、断点续传及秒传

文件分片上传、断点续传及秒传

功能介绍

文件上传小文件（图片、文档、视频）上传可以直接使用很多ui框架封装的上传组件，或者自己写一个input 上传，利用FormData 对象提交文件数据，后端使用spring提供的MultipartFile进行文件的接收，然后写入即可。但是对于比较大的文件，比如上传2G左右的文件（http上传），就需要将文件分片上传（file.slice()），否则中间http长时间连接可能会断掉。
分片上传分片上传，就是将所要上传的文件，按照一定的大小，将整个文件分隔成多个数据块（我们称之为Part）来进行分别上传，上传完之后再由服务端对所有上传的文件进行汇总整合成原始的文件。
秒传通俗的说，你把要上传的东西上传，服务器会先做MD5校验，如果服务器上有一样的东西，它就直接给你个新地址，其实你下载的都是服务器上的同一个文件，想要不秒传，其实只要让MD5改变，就是对文件本身做一下修改（改名字不行），例如一个文本文件，你多加几个字，MD5就变了，就不会秒传了.
断点续传断点续传是在下载或上传时，将下载或上传任务（一个文件或一个压缩包）人为的划分为几个部分，每一个部分采用一个线程进行上传或下载，如果碰到网络故障，可以从已经上传或下载的部分开始继续上传或者下载未完成的部分，而没有必要从头开始上传或者下载。本文的断点续传主要是针对断点上传场景。

大文件上传流程

前端对文件进行MD5加密，并且将文件按一定的规则分片
vue-simple-uploader先会发送get请求校验分片数据在服务端是否完整，如果完整则进行秒传，如果不完整或者无数据，则进行分片上传。
后台校验MD5值，根据上传的序号和分片大小计算相应的开始位置并写入该分片数据到文件中。

代码实现

web端

源码链接： https://gitee.com/KT1205529635/simple-uploader/tree/master/vue-uploader-master 本次参考了官方文档已经给位大佬的案例，根据自己的想法，实现了大文件的分片上传、断点续传及秒传其中前端写了三个案例

官方原生的案例修改
自己根据插件提供的api和钩子，自己diy自定义上传（配合springboot后台，文件夹上传未作处理）
自己diy自定义上传的基础上，在前端处理文件夹上传（文件夹只接收文件夹里的所有文件，未处理文件夹相对目录，可自己拓展）

官方原生的案例修改

效果如下

代码如下

VueUploader.vue https://gitee.com/KT1205529635/simple-uploader/blob/master/vue-uploader-master/src/views/VueUploader.vue#

代码语言：javascript复制

<template>
  <div class="container">
    <div class="logo"><img src="@/assets/logo.png" /></div>
    <uploader
      ref="uploader"
      :options="options"
      :autoStart="false"
      :file-status-text="fileStatusText"
      @file-added="onFileAdded"
      @file-success="onFileSuccess"
      @file-error="onFileError"
      @file-progress="onFileProgress"
      class="uploader-example"
    >
      <uploader-unsupport></uploader-unsupport>
      <uploader-drop>
        <p>拖动文件到这里上传</p>
        <uploader-btn>选择文件</uploader-btn>
        <uploader-btn :directory="true">选择文件夹</uploader-btn>
      </uploader-drop>
      <!-- uploader-list可自定义样式 -->
      <!-- <uploader-list></uploader-list> -->
      <uploader-list>
        <div class="file-panel" :class="{ collapse: collapse }">
          <div class="file-title">
            <p class="file-list-title">文件列表</p>
            <div class="operate">
              <el-button
                type="text"
                @click="operate"
                :title="collapse ? '折叠' : '展开'"
              >
                <i
                  class="icon"
                  :class="
                    collapse ? 'el-icon-caret-bottom' : 'el-icon-caret-top'
                  "
                ></i>
              </el-button>
              <el-button type="text" @click="close" title="关闭">
                <i class="icon el-icon-close"></i>
              </el-button>
            </div>
          </div>

          <ul
            class="file-list"
            :class="
              collapse ? 'uploader-list-ul-show' : 'uploader-list-ul-hidden'
            "
          >
            <li v-for="file in uploadFileList" :key="file.id">
              <uploader-file
                :class="'file_'   file.id"
                ref="files"
                :file="file"
                :list="true"
              ></uploader-file>
            </li>
            <div class="no-file" v-if="!uploadFileList.length">
              <i class="icon icon-empty-file"></i> 暂无待上传文件
            </div>
          </ul>
        </div>
      </uploader-list>
      <span>下载</span>
    </uploader>
  </div>
</template>

<script>
import SparkMD5 from "spark-md5";
const FILE_UPLOAD_ID_KEY = "file_upload_id";
// 分片大小，20MB
const CHUNK_SIZE = 20 * 1024 * 1024;
export default {
  data() {
    return {
      options: {
        // 上传地址
        target: "http://127.0.0.1:8025/api/upload",
        // 是否开启服务器分片校验。默认为 true
        testChunks: true,
        // 真正上传的时候使用的 HTTP 方法,默认 POST
        uploadMethod: "post",
        // 分片大小
        chunkSize: CHUNK_SIZE,
        // 并发上传数，默认为 3
        simultaneousUploads: 3,
        /**
         * 判断分片是否上传，秒传和断点续传基于此方法
         * 这里根据实际业务来 用来判断哪些片已经上传过了 不用再重复上传了 [这里可以用来写断点续传！！！]
         */
        checkChunkUploadedByResponse: (chunk, message) => {
          // message是后台返回
          let messageObj = JSON.parse(message);
          let dataObj = messageObj.data;
          if (dataObj.uploaded !== undefined) {
            return dataObj.uploaded;
          }
          // 判断文件或分片是否已上传，已上传返回 true
          // 这里的 uploadedChunks 是后台返回]
          return (dataObj.uploadedChunks || []).indexOf(chunk.offset   1) >= 0;
        },
        parseTimeRemaining: function (timeRemaining, parsedTimeRemaining) {
          //格式化时间
          return parsedTimeRemaining
            .replace(/syears?/, "年")
            .replace(/days?/, "天")
            .replace(/shours?/, "小时")
            .replace(/sminutes?/, "分钟")
            .replace(/sseconds?/, "秒");
        },
      },
      // 修改上传状态
      fileStatusTextObj: {
        success: "上传成功",
        error: "上传错误",
        uploading: "正在上传",
        paused: "停止上传",
        waiting: "等待中",
      },
      uploadIdInfo: null,
      uploadFileList: [],
      fileChunkList: [],
      collapse: true,
    };
  },
  created() {},
  methods: {
    onFileAdded(file, event) {
      console.log("file :>> ", file);
      // 有时 fileType为空，需截取字符
      console.log("文件类型："   file.fileType);
      // 文件大小
      console.log("文件大小："   file.size   "B");
      // 1. todo 判断文件类型是否允许上传
      // 2. 计算文件 MD5 并请求后台判断是否已上传，是则取消上传
      console.log("校验MD5");
      this.getFileMD5(file, (md5) => {
        if (md5 != "") {
          // 修改文件唯一标识
          file.uniqueIdentifier = md5;
          // 请求后台判断是否上传
          // 恢复上传
          file.resume();
        }
      });
    },
    onFileSuccess(rootFile, file, response, chunk) {
      this.uploadFileList = this.$refs.uploader.fileList;
      console.log(this.uploadFileList);
      console.log("上传成功");
    },
    onFileError(rootFile, file, message, chunk) {
      console.log("上传出错："   message);
    },
    onFileProgress(rootFile, file, chunk) {
      console.log(`当前进度：${Math.ceil(file._prevProgress * 100)}%`);
    },

    // 计算文件的MD5值
    getFileMD5(file, callback) {
      let spark = new SparkMD5.ArrayBuffer();
      let fileReader = new FileReader();
      //获取文件分片对象（注意它的兼容性，在不同浏览器的写法不同）
      let blobSlice =
        File.prototype.slice ||
        File.prototype.mozSlice ||
        File.prototype.webkitSlice;
      // 当前分片下标
      let currentChunk = 0;
      // 分片总数(向下取整)
      let chunks = Math.ceil(file.size / CHUNK_SIZE);
      // MD5加密开始时间
      let startTime = new Date().getTime();
      // 暂停上传
      file.pause();
      loadNext();
      // fileReader.readAsArrayBuffer操作会触发onload事件
      fileReader.onload = function (e) {
        // console.log("currentChunk :>> ", currentChunk);
        spark.append(e.target.result);
        if (currentChunk < chunks) {
          currentChunk  ;
          loadNext();
        } else {
          // 该文件的md5值
          let md5 = spark.end();
          console.log(
            `MD5计算完毕：${md5}，耗时：${new Date().getTime() - startTime} ms.`
          );
          // 回调传值md5
          callback(md5);
        }
      };
      fileReader.onerror = function () {
        this.$message.error("文件读取错误");
        file.cancel();
      };
      // 加载下一个分片
      function loadNext() {
        const start = currentChunk * CHUNK_SIZE;
        const end =
          start   CHUNK_SIZE >= file.size ? file.size : start   CHUNK_SIZE;
        // 文件分片操作，读取下一分片(fileReader.readAsArrayBuffer操作会触发onload事件)
        fileReader.readAsArrayBuffer(blobSlice.call(file.file, start, end));
      }
    },
    fileStatusText(status, response) {
      if (status === "md5") {
        return "校验MD5";
      } else {
        return this.fileStatusTextObj[status];
      }
    },
    /**
     * 折叠、展开面板动态切换
     */
    operate() {
      if (this.collapse === false) {
        this.collapse = true;
      } else {
        this.collapse = false;
      }
    },

    /**
     * 关闭折叠面板
     */
    close() {
      this.uploaderPanelShow = false;
    },
  },
};
</script>

<style lang="less" scoped>
.logo {
  font-family: "Avenir", Helvetica, Arial, sans-serif;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  text-align: center;
  color: #2c3e50;
  margin-top: 60px;
}
.uploader-example {
  width: 880px;
  padding: 15px;
  margin: 40px auto 0;
  font-size: 12px;
  box-shadow: 0 0 10px rgba(0, 0, 0, 0.4);
}
.uploader-example .uploader-btn {
  margin-right: 4px;
}
.uploader-example .uploader-list {
  max-height: 440px;
  overflow: auto;
  overflow-x: hidden;
  overflow-y: auto;
}

#global-uploader {
  position: fixed;
  z-index: 20;
  right: 15px;
  bottom: 15px;
  width: 550px;
}

.file-panel {
  background-color: #fff;
  border: 1px solid #e2e2e2;
  border-radius: 7px 7px 0 0;
  box-shadow: 0 0 10px rgba(0, 0, 0, 0.2);
}

.file-title {
  display: flex;
  height: 60px;
  line-height: 30px;
  padding: 0 15px;
  border-bottom: 1px solid #ddd;
}

.file-title {
  background-color: #e7ecf2;
}

.uploader-file-meta {
  display: none !important;
}

.operate {
  flex: 1;
  text-align: right;
}

.file-list {
  position: relative;
  height: 240px;
  overflow-x: hidden;
  overflow-y: auto;
  background-color: #fff;
  padding: 0px;
  margin: 0 auto;
  transition: all 0.5s;
}

.uploader-file-size {
  width: 15% !important;
}

.uploader-file-status {
  width: 32.5% !important;
  text-align: center !important;
}

li {
  background-color: #fff;
  list-style-type: none;
}

.no-file {
  position: absolute;
  top: 50%;
  left: 50%;
  transform: translate(-50%, -50%);
  font-size: 16px;
}

/* 隐藏上传按钮 */
.global-uploader-btn {
  display: none !important;
  clip: rect(0, 0, 0, 0);
  /* width: 100px;
  height: 50px; */
}

.file-list-title {
  /*line-height: 10px;*/
  font-size: 16px;
}

.uploader-file-name {
  width: 36% !important;
}

.uploader-file-actions {
  float: right !important;
}

.uploader-list-ul-hidden {
  height: 0px;
}
</style>

自定义uploader1

根据插槽和钩子函数，实现自定义插件样式，也实现简单的下载。

效果如下

代码如下

DiyUpload1.vue https://gitee.com/KT1205529635/simple-uploader/blob/master/vue-uploader-master/src/views/DiyUpload1.vue#

代码语言：javascript复制

<template>
  <div class="container">
    <div class="logo"><img src="@/assets/logo.png" /></div>
    <uploader
      ref="uploader"
      :options="options"
      :autoStart="false"
      :file-status-text="fileStatusText"
      @file-added="onFileAdded"
      @file-success="onFileSuccess"
      @file-error="onFileError"
      @file-progress="onFileProgress"
      class="uploader-example"
    >
      <uploader-unsupport></uploader-unsupport>
      <uploader-drop>
        <p>拖动文件到这里上传</p>
        <uploader-btn>选择文件</uploader-btn>
        <!-- <uploader-btn :directory="true">选择文件夹</uploader-btn> -->
      </uploader-drop>
      <!-- uploader-list可自定义样式 -->
      <!-- <uploader-list></uploader-list> -->
      <uploader-list>
        <div class="file-panel" :class="{ collapse: collapse }">
          <div class="file-title">
            <p class="file-list-title">文件列表</p>
            <div class="operate">
              <el-button
                type="text"
                @click="operate"
                :title="collapse ? '折叠' : '展开'"
              >
                <i
                  class="icon"
                  :class="
                    collapse ? 'el-icon-caret-bottom' : 'el-icon-caret-top'
                  "
                ></i>
              </el-button>
              <el-button type="text" @click="close" title="关闭">
                <i class="icon el-icon-close"></i>
              </el-button>
            </div>
          </div>

          <ul
            class="file-list"
            :class="
              collapse ? 'uploader-list-ul-show' : 'uploader-list-ul-hidden'
            "
          >
            <li v-for="file in uploadFileList" :key="file.id">
              <!-- <uploader-file
                :class="'file_'   file.id"
                ref="files"
                :file="file"
                :list="true"
              ></uploader-file> -->

              <uploader-file :file="file" :list="true" ref="uploaderFile">
                <template slot-scope="props">
                  <div class="filebox">
                    <p class="fileNameBox">
                      <span class="fileIcon"></span>
                      {{ file.name }}
                    </p>
                    <p class="fileProgressBox">
                      <el-progress
                        class="progressLength"
                        :stroke-width="18"
                        :percentage="
                          parseInt(
                            props.progress.toFixed(2) * 100 - 1 < 0
                              ? 0
                              : props.progress.toFixed(2) * 100
                          )
                        "
                      ></el-progress>
                      <span
                        class="statusBtn progressBtn"
                        v-if="!file.completed"
                        @click="pause(file)"
                        ><i
                          class="el-icon-video-pause"
                          v-if="!file.paused"
                          title="暂停"
                        ></i
                        ><i class="el-icon-video-play" v-else title="继续"></i
                      ></span>
                      <span
                        v-else
                        class="downloadBtn progressBtn"
                        @click="download(file)"
                        ><i class="el-icon-download" title="下载"></i
                      ></span>
                      <span class="cancelBtn progressBtn" @click="remove(file)"
                        ><i class="el-icon-error" title="删除"></i
                      ></span>
                    </p>
                    <p class="fileInfoBox" v-if="!file.completed">
                      <span class="fileInfoItem"
                        >速度：{{ props.formatedAverageSpeed }}</span
                      >
                      <span class="fileInfoItem"
                        >已上传：{{
                          (
                            parseFloat(props.formatedSize) * props.progress
                          ).toFixed(1)
                        }}/{{ props.formatedSize }}</span
                      >
                      <span class="fileInfoItem"
                        >剩余时间：{{ props.formatedTimeRemaining }}</span
                      >
                    </p>
                    <p class="fileInfoBoxSuccess" v-else>上传成功</p>
                  </div>
                </template>
              </uploader-file>
            </li>
            <div class="no-file" v-if="!uploadFileList.length">
              <i class="icon icon-empty-file"></i> 暂无待上传文件
            </div>
          </ul>
        </div>
      </uploader-list>
    </uploader>
  </div>
</template>

<script>
import SparkMD5 from "spark-md5";
const FILE_UPLOAD_ID_KEY = "file_upload_id";
// 分片大小，20MB
const CHUNK_SIZE = 20 * 1024 * 1024;
export default {
  data() {
    return {
      options: {
        // 上传地址
        target: "http://127.0.0.1:8025/api/upload",
        // 是否开启服务器分片校验。默认为 true
        testChunks: true,
        // 真正上传的时候使用的 HTTP 方法,默认 POST
        uploadMethod: "post",
        // 分片大小
        chunkSize: CHUNK_SIZE,
        // 并发上传数，默认为 3
        simultaneousUploads: 3,
        /**
         * 判断分片是否上传，秒传和断点续传基于此方法
         * 这里根据实际业务来 用来判断哪些片已经上传过了 不用再重复上传了 [这里可以用来写断点续传！！！]
         */
        checkChunkUploadedByResponse: (chunk, message) => {
          // message是后台返回
          let messageObj = JSON.parse(message);
          let dataObj = messageObj.data;
          if (dataObj.uploaded !== undefined) {
            return dataObj.uploaded;
          }
          // 判断文件或分片是否已上传，已上传返回 true
          // 这里的 uploadedChunks 是后台返回]
          return (dataObj.uploadedChunks || []).indexOf(chunk.offset   1) >= 0;
        },
        parseTimeRemaining: function (timeRemaining, parsedTimeRemaining) {
          //格式化时间
          return parsedTimeRemaining
            .replace(/syears?/, "年")
            .replace(/days?/, "天")
            .replace(/shours?/, "小时")
            .replace(/sminutes?/, "分钟")
            .replace(/sseconds?/, "秒");
        },
      },
      // 修改上传状态
      fileStatusTextObj: {
        success: "上传成功",
        error: "上传错误",
        uploading: "正在上传",
        paused: "停止上传",
        waiting: "等待中",
      },
      uploadIdInfo: null,
      uploadFileList: [],
      fileChunkList: [],
      collapse: true,
    };
  },
  created() {},
  methods: {
    onFileAdded(file, event) {
      this.uploadFileList.push(file);
      console.log("file :>> ", file);
      // 有时 fileType为空，需截取字符
      console.log("文件类型："   file.fileType);
      // 文件大小
      console.log("文件大小："   file.size   "B");
      // 1. todo 判断文件类型是否允许上传
      // 2. 计算文件 MD5 并请求后台判断是否已上传，是则取消上传
      console.log("校验MD5");
      this.getFileMD5(file, (md5) => {
        if (md5 != "") {
          // 修改文件唯一标识
          file.uniqueIdentifier = md5;
          // 请求后台判断是否上传
          // 恢复上传
          file.resume();
        }
      });
    },
    onFileSuccess(rootFile, file, response, chunk) {
      console.log("上传成功");
    },
    onFileError(rootFile, file, message, chunk) {
      console.log("上传出错："   message);
    },
    onFileProgress(rootFile, file, chunk) {
      console.log(`当前进度：${Math.ceil(file._prevProgress * 100)}%`);
    },

    // 计算文件的MD5值
    getFileMD5(file, callback) {
      let spark = new SparkMD5.ArrayBuffer();
      let fileReader = new FileReader();
      //获取文件分片对象（注意它的兼容性，在不同浏览器的写法不同）
      let blobSlice =
        File.prototype.slice ||
        File.prototype.mozSlice ||
        File.prototype.webkitSlice;
      // 当前分片下标
      let currentChunk = 0;
      // 分片总数(向下取整)
      let chunks = Math.ceil(file.size / CHUNK_SIZE);
      // MD5加密开始时间
      let startTime = new Date().getTime();
      // 暂停上传
      file.pause();
      loadNext();
      // fileReader.readAsArrayBuffer操作会触发onload事件
      fileReader.onload = function (e) {
        // console.log("currentChunk :>> ", currentChunk);
        spark.append(e.target.result);
        if (currentChunk < chunks) {
          currentChunk  ;
          loadNext();
        } else {
          // 该文件的md5值
          let md5 = spark.end();
          console.log(
            `MD5计算完毕：${md5}，耗时：${new Date().getTime() - startTime} ms.`
          );
          // 回调传值md5
          callback(md5);
        }
      };
      fileReader.onerror = function () {
        this.$message.error("文件读取错误");
        file.cancel();
      };
      // 加载下一个分片
      function loadNext() {
        const start = currentChunk * CHUNK_SIZE;
        const end =
          start   CHUNK_SIZE >= file.size ? file.size : start   CHUNK_SIZE;
        // 文件分片操作，读取下一分片(fileReader.readAsArrayBuffer操作会触发onload事件)
        fileReader.readAsArrayBuffer(blobSlice.call(file.file, start, end));
      }
    },
    fileStatusText(status, response) {
      if (status === "md5") {
        return "校验MD5";
      } else {
        return this.fileStatusTextObj[status];
      }
    },
    /**
     * 折叠、展开面板动态切换
     */
    operate() {
      if (this.collapse === false) {
        this.collapse = true;
      } else {
        this.collapse = false;
      }
    },

    /**
     * 关闭折叠面板
     */
    close() {
      this.uploaderPanelShow = false;
    },

    // 点击暂停
    pause(file, id) {
      console.log("file :>> ", file);
      if (file.paused) {
        file.resume();
      } else {
        file.pause();
      }
    },
    // 点击删除

    remove(file) {
      this.uploadFileList.findIndex((item, index) => {
        if (item.id === file.id) {
          this.$nextTick(() => {
            this.uploadFileList.splice(index, 1);
          });

          return;
        }
      });
    },

    // 点击下载
    download(file, id) {
      console.log("file:>> ", file);
      window.location.href = `http://127.0.0.1:8025/api/download/${file.uniqueIdentifier}/${file.name}`;
    },
  },
};
</script>

<style lang="less" scoped>
.logo {
  font-family: "Avenir", Helvetica, Arial, sans-serif;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  text-align: center;
  color: #2c3e50;
  margin-top: 60px;
}
.uploader-example {
  width: 880px;
  padding: 15px;
  margin: 40px auto 0;
  font-size: 12px;
  box-shadow: 0 0 10px rgba(0, 0, 0, 0.4);
}
.uploader-example .uploader-btn {
  margin-right: 4px;
}
.uploader-example .uploader-list {
  max-height: 440px;
  overflow: auto;
  overflow-x: hidden;
  overflow-y: auto;
}

#global-uploader {
  position: fixed;
  z-index: 20;
  right: 15px;
  bottom: 15px;
  width: 550px;
}

.file-panel {
  background-color: #fff;
  border: 1px solid #e2e2e2;
  border-radius: 7px 7px 0 0;
  box-shadow: 0 0 10px rgba(0, 0, 0, 0.2);
}

.file-title {
  display: flex;
  height: 60px;
  line-height: 30px;
  padding: 0 15px;
  border-bottom: 1px solid #ddd;
}

.file-title {
  background-color: #e7ecf2;
}
.uploader-file {
  height: 90px;
}

.uploader-file-meta {
  display: none !important;
}

.operate {
  flex: 1;
  text-align: right;
}

.file-list {
  position: relative;
  height: 300px;
  overflow-x: hidden;
  overflow-y: auto;
  background-color: #fff;
  padding: 0px;
  margin: 0 auto;
  transition: all 0.5s;
}

.uploader-file-size {
  width: 15% !important;
}

.uploader-file-status {
  width: 32.5% !important;
  text-align: center !important;
}

li {
  background-color: #fff;
  list-style-type: none;
}

.no-file {
  position: absolute;
  top: 50%;
  left: 50%;
  transform: translate(-50%, -50%);
  font-size: 16px;
}

.file-list-title {
  /*line-height: 10px;*/
  font-size: 16px;
}

.uploader-file-name {
  width: 36% !important;
}

.uploader-file-actions {
  float: right !important;
}

.uploader-list-ul-hidden {
  height: 0px;
}

.filebox {
  width: 100%;
  height: 60px;
}
.fileNameBox {
  width: 85%;
  margin: 0;
  padding: 0;
  font-size: 16px;
  margin-top: 5px;
  height: 30px;
  line-height: 30px;
  text-align: center;
}
.fileProgressBox {
  margin: 0;
  padding: 0;
  height: 20px;
  line-height: 20px;
  margin-top: 5px;
  margin-left: 10px;
  width: 100%;
}
/deep/ .el-progress-bar {
  width: 95%;
}
.progressLength {
  display: inline-block;
  line-height: 20px;
  width: 80%;
}
.progressBtn {
  margin-top: -5px;
  position: absolute;
  display: inline-block;
  font-size: 36px;
  margin-left: 10px;
  cursor: pointer;
}
.statusBtn {
  right: 90px;
  color: #ffba00;
}
.statusBtn:hover {
  color: #ffc833;
}
.cancelBtn {
  right: 30px;
  color: #ff4949;
}
.cancelBtn {
  margin-left: 10px;
}
.cancelBtn:hover {
  color: #ff6d6d;
}
.downloadBtn {
  right: 90px;
  color: #67c23a;
}
.downloadBtn:hover {
  color: #85ce61;
}
.fileInfoBox {
  margin: 0;
  padding: 0;
  font-size: 16px;
  width: 100%;
  height: 30px;
  line-height: 30px;
  margin-left: 10px;
  margin-bottom: 5px;

  .fileInfoItem {
    display: inline-block;
    width: 33%;
  }
}
.fileInfoBoxSuccess {
  margin: 0;
  padding: 0;
  font-size: 16px;
  width: 85%;
  height: 30px;
  line-height: 30px;
  margin-bottom: 5px;
  text-align: center;
}
</style>

自定义uploader2

在自定义uploader1上实现可上传文件夹

效果如下

代码如下

https://gitee.com/KT1205529635/simple-uploader/blob/master/vue-uploader-master/src/views/DiyUpload2.vue#

SpringBoot实现后端

源码链接： https://gitee.com/KT1205529635/simple-uploader/tree/master/springboot-upload-master 后端实现简单粗暴：springboot jpa hutool mysql 主要实现：

get请求接口校验上传文件MD5值和文件是否完整
post请求接收上传文件，并且计算分片，写入合成文件
文件完整上传完成时，往文件存储表tool_local_storage中加一条该文件的信息
get请求接口实现简单的文件下载

目录结构如下：

关键代码如下：

sql如下

代码语言：javascript复制

DROP TABLE IF EXISTS `file_chunk`;
CREATE TABLE `file_chunk`  (
`id` bigint(20) UNSIGNED NOT NULL AUTO_INCREMENT,
`file_name` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT '文件名',
`chunk_number` int(11) NULL DEFAULT NULL COMMENT '当前分片，从1开始',
`chunk_size` float NULL DEFAULT NULL COMMENT '分片大小',
`current_chunk_size` float NULL DEFAULT NULL COMMENT '当前分片大小',
`total_size` double(20, 0) NULL DEFAULT NULL COMMENT '文件总大小',
`total_chunk` int(11) NULL DEFAULT NULL COMMENT '总分片数',
`identifier` varchar(45) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT '文件标识',
`relative_path` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT 'md5校验码',
`createtime` timestamp(0) NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updatetime` timestamp(0) NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP(0),
PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 1529 CHARACTER SET = utf8 COLLATE = utf8_general_ci ROW_FORMAT = Dynamic;
DROP TABLE IF EXISTS `tool_local_storage`;
CREATE TABLE `tool_local_storage`  (
`id` bigint(20) NOT NULL AUTO_INCREMENT COMMENT 'ID',
`real_name` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT '文件真实的名称',
`name` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT '文件名',
`suffix` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT '后缀',
`path` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT '路径',
`type` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT '类型',
`size` varchar(100) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT '大小',
`identifier` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT 'md5校验码rn',
`create_by` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT '创建者',
`update_by` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci NULL DEFAULT NULL COMMENT '更新者',
`createtime` timestamp(0) NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updatetime` timestamp(0) NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP(0),
PRIMARY KEY (`id`) USING BTREE
) ENGINE = InnoDB AUTO_INCREMENT = 3360 CHARACTER SET = utf8 COLLATE = utf8_general_ci COMMENT = '文件存储' ROW_FORMAT = Compact;

controller实现

代码语言：javascript复制

package cn.kt.springbootuploadmaster.controller;
import cn.kt.springbootuploadmaster.domin.FileChunkParam;
import cn.kt.springbootuploadmaster.domin.ResultVO;
import cn.kt.springbootuploadmaster.service.FileChunkService;
import cn.kt.springbootuploadmaster.service.FileService;
import cn.kt.springbootuploadmaster.service.LocalStorageService;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.*;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
/**
* Created by tao.
* Date: 2022/6/29 11:56
* 描述:
*/
@RestController
@Slf4j
@RequestMapping("/api")
public class FileUploadController {
@Autowired
private FileService fileService;
@Autowired
private FileChunkService fileChunkService;
@Autowired
private LocalStorageService localStorageService;
 
@GetMapping("/upload")
public ResultVO<Map<String, Object>> checkUpload(FileChunkParam param) {
    log.info("文件MD5:"   param.getIdentifier());
    List<FileChunkParam> list = fileChunkService.findByMd5(param.getIdentifier());
    Map<String, Object> data = new HashMap<>(1);
    // 判断文件存不存在
    if (list.size() == 0) {
        data.put("uploaded", false);
        return new ResultVO<>(200, "上传成功", data);
    }
    // 处理单文件
    if (list.get(0).getTotalChunks() == 1) {
        data.put("uploaded", true);
        data.put("url", "");
        return new ResultVO<Map<String, Object>>(200, "上传成功", data);
    }
    // 处理分片
    int[] uploadedFiles = new int[list.size()];
    int index = 0;
    for (FileChunkParam fileChunkItem : list) {
        uploadedFiles[index] = fileChunkItem.getChunkNumber();
        index  ;
    }
    data.put("uploadedChunks", uploadedFiles);
    return new ResultVO<Map<String, Object>>(200, "上传成功", data);
}
 
@PostMapping("/upload")
public ResultVO chunkUpload(FileChunkParam param) {
    log.info("上传文件：{}", param);
    boolean flag = fileService.uploadFile(param);
    if (!flag) {
        return new ResultVO(211, "上传失败");
    }
    return new ResultVO(200, "上传成功");
}
@GetMapping(value = "/download/{md5}/{name}")
public void downloadbyname(HttpServletRequest request, HttpServletResponse response, @PathVariable String name, @PathVariable String md5) throws IOException {
    localStorageService.downloadByName(name, md5, request, response);
}
}

FileService实现 FileService.java

代码语言：javascript复制

package cn.kt.springbootuploadmaster.service;
import cn.kt.springbootuploadmaster.domin.FileChunkParam;
/**
* Created by tao.
* Date: 2022/6/29 11:22
* 描述:
*/
public interface FileService {
/**
 * 上传文件
 * @param param 参数
 * @return
 */
boolean uploadFile(FileChunkParam param);
}

FileServiceImpl.java

代码语言：javascript复制

package cn.kt.springbootuploadmaster.service.impl;
import cn.kt.springbootuploadmaster.domin.FileChunkParam;
import cn.kt.springbootuploadmaster.enums.MessageEnum;
import cn.kt.springbootuploadmaster.exception.BusinessException;
import cn.kt.springbootuploadmaster.repository.LocalStorageRepository;
import cn.kt.springbootuploadmaster.service.FileChunkService;
import cn.kt.springbootuploadmaster.service.FileService;
import cn.kt.springbootuploadmaster.service.LocalStorageService;
import cn.kt.springbootuploadmaster.utils.FileUtil;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;
import sun.misc.Cleaner;
import java.io.File;
import java.io.IOException;
import java.io.RandomAccessFile;
import java.lang.reflect.Method;
import java.nio.MappedByteBuffer;
import java.nio.channels.FileChannel;
import java.security.AccessController;
import java.security.PrivilegedAction;
/**
* Created by tao.
* Date: 2022/6/29 11:22
* 描述:
*/
@Service("fileService")
@Slf4j
public class FileServiceImpl implements FileService {
/**
 * 默认的分片大小：20MB
 */
public static final long DEFAULT_CHUNK_SIZE = 20 * 1024 * 1024;
 
@Value("${file.BASE_FILE_SAVE_PATH}")
private String BASE_FILE_SAVE_PATH;
 
@Autowired
private FileChunkService fileChunkService;
 
@Autowired
private LocalStorageService localStorageService;
 
@Override
public boolean uploadFile(FileChunkParam param) {
    if (null == param.getFile()) {
        throw new BusinessException(MessageEnum.UPLOAD_FILE_NOT_NULL);
    }
    // 判断目录是否存在，不存在则创建目录
    File savePath = new File(BASE_FILE_SAVE_PATH);
    if (!savePath.exists()) {
        boolean flag = savePath.mkdirs();
        if (!flag) {
            log.error("保存目录创建失败");
            return false;
        }
    }
    //  todo 处理文件夹上传（上传目录下新建上传的文件夹）
    /*String relativePath = param.getRelativePath();
    if (relativePath.contains("/") || relativePath.contains(File.separator)) {
        String div = relativePath.contains(File.separator) ? File.separator : "/";
        String tempPath = relativePath.substring(0, relativePath.lastIndexOf(div));
        savePath = new File(BASE_FILE_SAVE_PATH   File.separator   tempPath);
        if (!savePath.exists()) {
            boolean flag = savePath.mkdirs();
            if (!flag) {
                log.error("保存目录创建失败");
                return false;
            }
        }
    }*/
    // 这里可以使用 uuid 来指定文件名，上传完成后再重命名，File.separator指文件目录分割符，win上的"",Linux上的"/"。
    String fullFileName = savePath   File.separator   param.getFilename();
    // 单文件上传
    if (param.getTotalChunks() == 1) {
        return uploadSingleFile(fullFileName, param);
    }
    // 分片上传，这里使用 uploadFileByRandomAccessFile 方法，也可以使用 uploadFileByMappedByteBuffer 方法上传
    boolean flag = uploadFileByRandomAccessFile(fullFileName, param);
    if (!flag) {
        return false;
    }
    // 保存分片上传信息
    fileChunkService.saveFileChunk(param);
    return true;
}
private boolean uploadFileByRandomAccessFile(String resultFileName, FileChunkParam param) {
    try (RandomAccessFile randomAccessFile = new RandomAccessFile(resultFileName, "rw")) {
        // 分片大小必须和前端匹配，否则上传会导致文件损坏
        long chunkSize = param.getChunkSize() == 0L ? DEFAULT_CHUNK_SIZE : param.getChunkSize().longValue();
        // 偏移量
        long offset = chunkSize * (param.getChunkNumber() - 1);
        // 定位到该分片的偏移量
        randomAccessFile.seek(offset);
        // 写入
        randomAccessFile.write(param.getFile().getBytes());
    } catch (IOException e) {
        log.error("文件上传失败："   e);
        return false;
    }
    return true;
}
 
private boolean uploadFileByMappedByteBuffer(String resultFileName, FileChunkParam param) {
    // 分片上传
    try (RandomAccessFile randomAccessFile = new RandomAccessFile(resultFileName, "rw");
         FileChannel fileChannel = randomAccessFile.getChannel()) {
        // 分片大小必须和前端匹配，否则上传会导致文件损坏
        long chunkSize = param.getChunkSize() == 0L ? DEFAULT_CHUNK_SIZE : param.getChunkSize().longValue();
        // 写入文件
        long offset = chunkSize * (param.getChunkNumber() - 1);
        byte[] fileBytes = param.getFile().getBytes();
        MappedByteBuffer mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_WRITE, offset, fileBytes.length);
        mappedByteBuffer.put(fileBytes);
        // 释放
        unmap(mappedByteBuffer);
    } catch (IOException e) {
        log.error("文件上传失败："   e);
        return false;
    }
    return true;
}
 
private boolean uploadSingleFile(String resultFileName, FileChunkParam param) {
    File saveFile = new File(resultFileName);
    try {
        // 写入
        param.getFile().transferTo(saveFile);
        localStorageService.saveLocalStorage(param);
    } catch (IOException e) {
        log.error("文件上传失败："   e);
        return false;
    }
    return true;
}
 
/**
 * 释放 MappedByteBuffer
 * 在 MappedByteBuffer 释放后再对它进行读操作的话就会引发 jvm crash，在并发情况下很容易发生
 * 正在释放时另一个线程正开始读取，于是 crash 就发生了。所以为了系统稳定性释放前一般需要检
 * 查是否还有线程在读或写
 * 来源：https://my.oschina.net/feichexia/blog/212318
 *
 * @param mappedByteBuffer mappedByteBuffer
 */
public static void unmap(final MappedByteBuffer mappedByteBuffer) {
    try {
        if (mappedByteBuffer == null) {
            return;
        }
        mappedByteBuffer.force();
        AccessController.doPrivileged((PrivilegedAction<Object>) () -> {
            try {
                Method getCleanerMethod = mappedByteBuffer.getClass()
                        .getMethod("cleaner");
                getCleanerMethod.setAccessible(true);
                Cleaner cleaner =
                        (Cleaner) getCleanerMethod
                                .invoke(mappedByteBuffer, new Object[0]);
                cleaner.clean();
            } catch (Exception e) {
                log.error("MappedByteBuffer 释放失败："   e);
            }
            System.out.println("clean MappedByteBuffer completed");
            return null;
        });
    } catch (Exception e) {
        log.error("unmap error:"   e);
    }
}
}

其他实现的细节可自己查看源码，也可以根据自己的想法在这个demo中进行拓展。理清楚其中的大文件传输、秒传、断点续传后，自己开发一个小网盘也不是什么难事了 ^_^

源码下载

https://gitee.com/KT1205529635/simple-uploader

https 网络安全 vue.js spark 对象存储

0 人点赞

Spring学习笔记（三十六）——SpringBoot 实现大文件分片上传、断点续传及秒传

文件分片上传、断点续传及秒传

功能介绍

相关插件技术介绍

大文件上传流程

代码实现

web端

官方原生的案例修改

效果如下

代码如下

自定义uploader1

效果如下

代码如下

自定义uploader2

效果如下

代码如下

SpringBoot实现后端

源码下载