代碼網(wǎng)站怎么做的關(guān)鍵詞優(yōu)化軟件排行
前言
大文件分片上傳和斷點(diǎn)續(xù)傳是為了解決在網(wǎng)絡(luò)傳輸過程中可能遇到的問題，以提高文件傳輸?shù)男屎头€(wěn)定性。
- 首先，大文件分片上傳是將大文件分割成較小的片段進(jìn)行上傳。這樣做的好處是可以減少單個(gè)文件的傳輸時(shí)間，因?yàn)檩^小的文件片段更容易快速上傳到目標(biāo)服務(wù)器。同時(shí)，如果在傳輸過程中出現(xiàn)錯(cuò)誤或中斷，只需要重新上傳出現(xiàn)問題的文件片段，而不需要重新上傳整個(gè)文件，從而減少了傳輸?shù)臅r(shí)間和帶寬消耗。
- 其次，斷點(diǎn)續(xù)傳是指在文件傳輸過程中，如果傳輸被中斷或者發(fā)生錯(cuò)誤，可以從上一次中斷的地方繼續(xù)傳輸，而不是從頭開始。這對(duì)于大文件的傳輸尤為重要，因?yàn)閭鬏斠粋€(gè)大文件可能需要較長(zhǎng)的時(shí)間，而中斷可能是由網(wǎng)絡(luò)問題、電源故障、軟件崩潰或其他因素引起的。斷點(diǎn)續(xù)傳功能允許用戶在中斷后恢復(fù)傳輸，而無(wú)需重新開始，節(jié)省了時(shí)間和資源。
大文件分片上傳和斷點(diǎn)續(xù)傳在以下情況下尤為重要：
- 低帶寬網(wǎng)絡(luò)環(huán)境：在網(wǎng)絡(luò)速度較慢或不穩(wěn)定的情況下，將大文件分割為較小的片段進(jìn)行上傳可以降低傳輸?shù)臅r(shí)間和失敗的風(fēng)險(xiǎn)。
- 大文件傳輸：對(duì)于大文件，一次性完整上傳可能需要很長(zhǎng)時(shí)間，而且中途出現(xiàn)問題時(shí)需要重新傳輸整個(gè)文件，因此將文件分割并實(shí)現(xiàn)斷點(diǎn)續(xù)傳功能可以提高效率和可靠性。
- 網(wǎng)絡(luò)中斷或傳輸錯(cuò)誤：網(wǎng)絡(luò)中斷、電源故障或軟件崩潰等因素可能導(dǎo)致文件傳輸中斷，斷點(diǎn)續(xù)傳功能可以從中斷處恢復(fù)，避免重新傳輸整個(gè)文件。
- 多用戶并發(fā)上傳：在有多個(gè)用戶同時(shí)上傳文件的情況下，分片上傳和斷點(diǎn)續(xù)傳可以減少對(duì)服務(wù)器資源的占用，提高并發(fā)傳輸?shù)男省?/li>
前端
采用百度的webuploader，在file.html中引用webuploader.js、jquery.js
代碼如下：
<!DOCTYPE html>
<html lang="en">
<head><meta charset="UTF-8"><title>大文件上傳下載</title><link rel="stylesheet" type="text/css" href="webuploader.css"><script src="jquery.js"></script><script src="webuploader.js"></script><style>#upload-container {width: 100px;height: 50px;background: #94d3e7;padding-bottom: 10px;}</style>
</head>
<body><div id="upload-container"><span>文件拖拽上傳</span></div><button id="picker" style="margin-top: 20px">分片上傳</button><div id="upload-list"></div><hr/>
<a href="/file/download" >普通下載</a>
<hr/>
<a href="/file/downloads" target="_blank">分片下載</a></body>
<script>$('#upload-container').click(function (event) {$("#picker").find('input').click();});// 初始化上傳組件const uploader = WebUploader.create({auto: true,swf: 'Uploader.swf', // swf文件路徑server: '/file/upload', // 上傳接口dnd: '#upload-container',pick: '#picker', // 內(nèi)部根據(jù)當(dāng)前運(yùn)行創(chuàng)建multiple: true, // 選擇多個(gè)chunked: true, // 開啟分片threads: 8, // 并發(fā)數(shù)，默認(rèn) 3chunkRetry: 8, // 如果遇到網(wǎng)絡(luò)錯(cuò)誤,重新上傳次數(shù)method: 'POST',fileSizeLimit: 1024 * 1024 * 1024 * 10, // 文件總大小為10GfileSingleSizeLimit: 1024 * 1024 * 1024 * 1, // 單個(gè)文件大小最大為1GfileVal: 'upload'});// 入隊(duì)之前觸發(fā)事件uploader.on("beforeFileQueued", function (file) {// 獲取文件后綴console.log(file.name);});// 當(dāng)有文件被添加進(jìn)隊(duì)列的時(shí)候uploader.on('fileQueued', function (file) {$('#upload-list').append( '<div id="' + file.id + '" class="item">' +'<h4 class="info">' + file.name + '</h4>' +'<p class="state">等待上傳...</p>' +'</div>' );});// 文件上傳過程中創(chuàng)建進(jìn)度條實(shí)時(shí)顯示。uploader.on('uploadProgress', function (file, percentage) {var $li = $('#' + file.id),$percent = $li.find('.progress .progress-bar');// 避免重復(fù)創(chuàng)建if (!$percent.length) {$percent = $('<div class="progress progress-striped active">' +'<div class="progress-bar" role="progressbar" style="width: 0%">' +'</div>' +'</div>').appendTo($li).find('.progress-bar');}$li.find('p.state').text('上傳中');$percent.css('width', percentage * 100 + '%');});uploader.on( 'uploadSuccess', function( file ) {$( '#'+file.id ).find('p.state').text('已上傳');});uploader.on( 'uploadError', function( file ) {$( '#'+file.id ).find('p.state').text('上傳出錯(cuò)');});uploader.on( 'uploadComplete', function( file ) {$( '#'+file.id ).find('.progress').fadeOut();});</script>
</html>
后端
1.Pom文件添加依賴
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd"><modelVersion>4.0.0</modelVersion><parent><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-parent</artifactId><version>2.7.4</version><relativePath/></parent><groupId>com.liyh</groupId><artifactId>springboot-file</artifactId><version>0.0.1</version><name>springboot-file</name><description>Demo project for Spring Boot</description><properties><java.version>1.8</java.version></properties><dependencies><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-web</artifactId></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-test</artifactId><scope>test</scope></dependency><dependency><groupId>commons-fileupload</groupId><artifactId>commons-fileupload</artifactId><version>1.3.1</version></dependency><dependency><groupId>commons-io</groupId><artifactId>commons-io</artifactId><version>2.4</version></dependency><!-- 做斷點(diǎn)下載使用 --><dependency><groupId>org.apache.httpcomponents</groupId><artifactId>httpcore</artifactId></dependency><dependency><groupId>org.apache.httpcomponents</groupId><artifactId>httpclient</artifactId></dependency><dependency><groupId>org.projectlombok</groupId><artifactId>lombok</artifactId><version>1.18.22</version></dependency><dependency><groupId>org.apache.commons</groupId><artifactId>commons-lang3</artifactId><version>3.12.0</version></dependency></dependencies><build><plugins><plugin><groupId>org.springframework.boot</groupId><artifactId>spring-boot-maven-plugin</artifactId></plugin></plugins></build></project>
2.yml配置文件
# 配置服務(wù)端口
server:port: 8018spring:servlet:multipart:# Spring Boot中有默認(rèn)的文件上傳組件，在使用ServletFileUpload時(shí)需要關(guān)閉Spring Boot的默認(rèn)配置enabled: false# 設(shè)置單個(gè)文件大小max-file-size: 1GB# 設(shè)置單次請(qǐng)求文件的總大小max-request-size: 10GB
3..編寫測(cè)試接口controller
package com.liyh.controller;import com.liyh.service.FileService;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;/*** 文件上傳測(cè)試接口** @author liyh*/
@RestController
@RequestMapping("/file")
public class FileController {@Autowiredprivate FileService fileService;/*** 單個(gè)文件上傳，支持?jǐn)帱c(diǎn)續(xù)傳*/@PostMapping("/upload")public void upload(HttpServletRequest request, HttpServletResponse response) {try {fileService.upload(request, response);} catch (Exception e) {e.printStackTrace();}}/*** 普通文件下載*/@GetMapping("/download")public void download(HttpServletRequest request, HttpServletResponse response) throws IOException {fileService.download(request, response);}/*** 分片文件下載*/@GetMapping("/downloads")public String downloads() throws IOException {fileService.downloads();return "下載成功";}}
4.編寫service?
package com.liyh.service;import com.liyh.entity.DownloadFileInfo;
import com.liyh.entity.FileInfo;
import com.liyh.entity.UploadFileInfo;
import org.apache.commons.fileupload.FileItem;
import org.apache.commons.fileupload.disk.DiskFileItemFactory;
import org.apache.commons.fileupload.servlet.ServletFileUpload;
import org.apache.commons.io.FileUtils;
import org.apache.commons.lang3.StringUtils;
import org.apache.http.HttpEntity;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.impl.client.HttpClients;
import org.springframework.stereotype.Service;import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.*;
import java.net.URLDecoder;
import java.net.URLEncoder;
import java.util.List;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;@Service
public class FileService {/*** 編碼*/private static final String UTF_8 = "UTF-8";/*** 文件上傳路徑(當(dāng)前項(xiàng)目路徑下，也可配置固定路徑)*/private String uploadPath = System.getProperty("user.dir") + "/springboot-file/upload/";/*** 下載指定文件*/private String downloadFile = "D:\\Download\\git.exe";/*** 文件下載地址(當(dāng)前項(xiàng)目路徑下，也可配置固定路徑)*/private String downloadPath = System.getProperty("user.dir") + "/springboot-file/download/";/*** 分片下載每一片大小為50M*/private static final Long PER_SLICE = 1024 * 1024 * 50L;/*** 定義分片下載線程池*/private ExecutorService executorService = Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());/*** final string*/private static final String RANGE = "Range";/*** 上傳文件*/public void upload(HttpServletRequest request, HttpServletResponse response) throws Exception {// 獲取ServletFileUploadServletFileUpload servletFileUpload = getServletFileUpload();List<FileItem> items = servletFileUpload.parseRequest(request);// 獲取文件信息UploadFileInfo uploadFileInfo = getFileInfo(items);// 寫入臨時(shí)文件writeTempFile(items, uploadFileInfo);// 判斷是否合并mergeFile(uploadFileInfo);// 返回結(jié)果response.setCharacterEncoding(UTF_8);response.getWriter().write("上傳成功");}/*** 獲取ServletFileUpload*/private ServletFileUpload getServletFileUpload() {// 設(shè)置緩沖區(qū)大小，先讀到內(nèi)存里在從內(nèi)存寫DiskFileItemFactory factory = new DiskFileItemFactory();factory.setSizeThreshold(1024);File file = new File(uploadPath);// 如果文件夾不存在則創(chuàng)建if (!file.exists() && !file.isDirectory()) {file.mkdirs();}factory.setRepository(file);// 解析ServletFileUpload upload = new ServletFileUpload(factory);// 設(shè)置單個(gè)大小與最大大小upload.setFileSizeMax(1 * 1024 * 1024 * 1024L);upload.setSizeMax(10 * 1024 * 1024 * 1024L);return upload;}/*** 獲取文件信息** @param items* @return* @throws UnsupportedEncodingException*/private UploadFileInfo getFileInfo(List<FileItem> items) throws UnsupportedEncodingException {UploadFileInfo uploadFileInfo = new UploadFileInfo();for (FileItem item : items) {if (item.isFormField()) {// 獲取分片數(shù)據(jù)if ("chunk".equals(item.getFieldName())) {uploadFileInfo.setCurrentChunk(Integer.parseInt(item.getString(UTF_8)));}if ("chunks".equals(item.getFieldName())) {uploadFileInfo.setChunks(Integer.parseInt(item.getString(UTF_8)));}if ("name".equals(item.getFieldName())) {uploadFileInfo.setFileName(item.getString(UTF_8));}}}return uploadFileInfo;}/*** 寫入臨時(shí)文件** @param items* @param uploadFileInfo* @throws Exception*/private void writeTempFile(List<FileItem> items, UploadFileInfo uploadFileInfo) throws Exception {// 獲取文件基本信息后for (FileItem item : items) {if (!item.isFormField()) {// 有分片需要臨時(shí)目錄String tempFileName = uploadFileInfo.getFileName();if (StringUtils.isNotBlank(tempFileName)) {if (uploadFileInfo.getCurrentChunk() != null) {tempFileName = uploadFileInfo.getCurrentChunk() + "_" + uploadFileInfo.getFileName();}// 判斷文件是否存在File tempFile = new File(uploadPath, tempFileName);// 斷點(diǎn)續(xù)傳，判斷文件是否存在，若存在則不傳if (!tempFile.exists()) {item.write(tempFile);}}}}}/*** 判斷是否合并** @param uploadFileInfo* @throws IOException* @throws InterruptedException*/private void mergeFile(UploadFileInfo uploadFileInfo) throws IOException, InterruptedException {Integer currentChunk = uploadFileInfo.getCurrentChunk();Integer chunks = uploadFileInfo.getChunks();String fileName = uploadFileInfo.getFileName();// 如果當(dāng)前分片等于總分片那么合并文件if (currentChunk != null && chunks != null && currentChunk.equals(chunks - 1)) {File tempFile = new File(uploadPath, fileName);try (BufferedOutputStream os = new BufferedOutputStream(new FileOutputStream(tempFile))) {// 根據(jù)之前命名規(guī)則找到所有分片for (int i = 0; i < chunks; i++) {File file = new File(uploadPath, i + "_" + fileName);// 并發(fā)情況，需要判斷所有，因?yàn)榭赡茏詈笠粋€(gè)分片傳完，之前有的還沒傳完while (!file.exists()) {// 不存在休眠100毫秒后在重新判斷Thread.sleep(100);}// 分片存在，讀入數(shù)組中byte[] bytes = FileUtils.readFileToByteArray(file);os.write(bytes);os.flush();file.delete();}os.flush();}}}/*** 文件下載** @param request* @param response* @throws IOException*/public void download(HttpServletRequest request, HttpServletResponse response) throws IOException {// 獲取文件File file = new File(downloadFile);// 獲取下載文件信息DownloadFileInfo downloadFileInfo = getDownloadFileInfo(file.length(), request, response);// 設(shè)置響應(yīng)頭setResponse(response, file.getName(), downloadFileInfo);// 下載文件try (InputStream is = new BufferedInputStream(new FileInputStream(file));OutputStream os = new BufferedOutputStream(response.getOutputStream())) {// 跳過已經(jīng)讀取文件is.skip(downloadFileInfo.getPos());byte[] buffer = new byte[1024];long sum = 0;// 讀取while (sum < downloadFileInfo.getRangeLength()) {int length = is.read(buffer, 0, (downloadFileInfo.getRangeLength() - sum) <= buffer.length ? (int) (downloadFileInfo.getRangeLength() - sum) : buffer.length);sum = sum + length;os.write(buffer, 0, length);}}}/*** 有兩個(gè)map，我要去判斷里面相同鍵的值一致不一致，除了雙重for循環(huán)，有沒有別的好辦法*/private DownloadFileInfo getDownloadFileInfo(long fSize, HttpServletRequest request, HttpServletResponse response) {long pos = 0;long last = fSize - 1;// 判斷前端是否需要分片下載if (request.getHeader(RANGE) != null) {response.setStatus(HttpServletResponse.SC_PARTIAL_CONTENT);String numRange = request.getHeader(RANGE).replace("bytes=", "");String[] strRange = numRange.split("-");if (strRange.length == 2) {pos = Long.parseLong(strRange[0].trim());last = Long.parseLong(strRange[1].trim());// 若結(jié)束字節(jié)超出文件大小，取文件大小if (last > fSize - 1) {last = fSize - 1;}} else {// 若只給一個(gè)長(zhǎng)度，開始位置一直到結(jié)束pos = Long.parseLong(numRange.replace("-", "").trim());}}long rangeLength = last - pos + 1;String contentRange = "bytes " + pos + "-" + last + "/" + fSize;return new DownloadFileInfo(fSize, pos, last, rangeLength, contentRange);}/*** 分片下載** @throws IOException*/public void downloads() throws IOException {File file = new File(downloadPath);// 如果文件夾不存在則創(chuàng)建if (!file.exists() && !file.isDirectory()) {file.mkdirs();}// 探測(cè)下載，獲取文件相關(guān)信息FileInfo fileInfoDto = sliceDownload(1, 10, -1, null);// 如果不為空，執(zhí)行分片下載if (fileInfoDto != null) {// 計(jì)算有多少分片long pages = fileInfoDto.getFileSize() / PER_SLICE;// 適配最后一個(gè)分片for (long i = 0; i <= pages; i++) {long start = i * PER_SLICE;long end = (i + 1) * PER_SLICE - 1;executorService.execute(new SliceDownloadRunnable(start, end, i, fileInfoDto.getFileName()));}}}/*** 分片下載** @param start 分片起始位置* @param end 分片結(jié)束位置* @param page 第幾個(gè)分片, page=-1時(shí)是探測(cè)下載*/private FileInfo sliceDownload(long start, long end, long page, String fName) throws IOException {// 斷點(diǎn)下載File file = new File(downloadPath, page + "-" + fName);// 如果當(dāng)前文件已經(jīng)存在，并且不是探測(cè)任務(wù)，并且文件的長(zhǎng)度等于分片的大小，那么不用下載當(dāng)前文件if (file.exists() && page != -1 && file.length() == PER_SLICE) {return null;}// 創(chuàng)建HttpClientHttpClient client = HttpClients.createDefault();HttpGet httpGet = new HttpGet("http://localhost:8018/file/download");httpGet.setHeader(RANGE, "bytes=" + start + "-" + end);HttpResponse httpResponse = client.execute(httpGet);String fSize = httpResponse.getFirstHeader("fSize").getValue();fName = URLDecoder.decode(httpResponse.getFirstHeader("fName").getValue(), UTF_8);HttpEntity entity = httpResponse.getEntity();// 下載try (InputStream is = entity.getContent();FileOutputStream fos = new FileOutputStream(file)) {byte[] buffer = new byte[1024];int ch;while ((ch = is.read(buffer)) != -1) {fos.write(buffer, 0, ch);}fos.flush();}// 判斷是否是最后一個(gè)分片，如果是那么合并if (end - Long.parseLong(fSize) > 0) {mergeFile(fName, page);}return new FileInfo(Long.parseLong(fSize), fName);}private void mergeFile(String fName, long page) throws IOException {File file = new File(downloadPath, fName);try (BufferedOutputStream os = new BufferedOutputStream(new FileOutputStream(file))) {for (int i = 0; i <= page; i++) {File tempFile = new File(downloadPath, i + "-" + fName);// 文件不存在或文件沒寫完while (!tempFile.exists() || (i != page && tempFile.length() < PER_SLICE)) {Thread.sleep(100);}byte[] bytes = FileUtils.readFileToByteArray(tempFile);os.write(bytes);os.flush();tempFile.delete();}// 刪除文件File f = new File(downloadPath, "-1" + "-null");if (f.exists()) {f.delete();}} catch (InterruptedException e) {e.printStackTrace();}}private class SliceDownloadRunnable implements Runnable {private final long start;private final long end;private final long page;private final String fName;private SliceDownloadRunnable(long start, long end, long page, String fName) {this.start = start;this.end = end;this.page = page;this.fName = fName;}@Overridepublic void run() {try {sliceDownload(start, end, page, fName);} catch (IOException e) {e.printStackTrace();}}}/*** 設(shè)置響應(yīng)頭*/private void setResponse(HttpServletResponse response, String fileName, DownloadFileInfo downloadFileInfo) throws UnsupportedEncodingException {response.setCharacterEncoding(UTF_8);response.setContentType("application/x-download");response.addHeader("Content-Disposition", "attachment;filename=" + URLEncoder.encode(fileName, UTF_8));// 支持分片下載response.setHeader("Accept-Range", "bytes");response.setHeader("fSize", String.valueOf(downloadFileInfo.getFSize()));response.setHeader("fName", URLEncoder.encode(fileName, UTF_8));// range響應(yīng)頭response.setHeader("Content-Range", downloadFileInfo.getContentRange());response.setHeader("Content-Length", String.valueOf(downloadFileInfo.getRangeLength()));}}
5..編寫上傳文件實(shí)體類、文件信息實(shí)體類和下載文件實(shí)體類
package com.liyh.entity;import lombok.Data;@Data
public class UploadFileInfo {/*** 文件名稱*/private String fileName;/*** 上傳文件會(huì)有多個(gè)分片，記錄當(dāng)前為那個(gè)分片*/private Integer currentChunk;/*** 總分片數(shù)*/private Integer chunks;}
package com.liyh.entity;import lombok.AllArgsConstructor;
import lombok.NoArgsConstructor;
import lombok.Data;@Data
@AllArgsConstructor
@NoArgsConstructor
public class FileInfo {private long fileSize;private String fileName;}
package com.liyh.entity;import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;@AllArgsConstructor
@NoArgsConstructor
@Data
public class DownloadFileInfo {/*** 文件總大小*/private long fSize;/*** 斷點(diǎn)起始位置*/private long pos;/*** 斷點(diǎn)結(jié)束位置*/private long last;/*** rang響應(yīng)*/private long rangeLength;/*** range響應(yīng)*/private String contentRange;}
運(yùn)行效果
