Android Netroid解析之——断点续传下载及有关问题修正

提到Netroid或许很多人不知道这个框架，但我如果说Volley想必没有人不知道吧。Netroid是一个基于Volley实现的Android Http库。提供执行网络请求、缓存返回结果、批量图片加载、大文件断点下载的常见Http交互功能，关于网络请求，图片加载没什么好说的，Volley已经有很多人解析过了，这里来说一下大文件断点下载。

关于大文件断点下载，网上也有很多实现的demo,为什么要单单说Netroid呢？因为Netroid断点续传不依赖数据库，我在网上看到过很多的断点续传的例子，无一例外都是依赖于数据库，包括DownloadManager，大名鼎鼎的xutils,但是这两个都有一定的问题。

1.DownloadManager在三星手机上必须打开下载管理才能应用，而打开这个管理必须需要手动打开，一般情况下无伤大雅，视情况而定

2.xutils这个框架别的不知道，文件下载这块慎用

好了，进入正题，Netroid的地址:https://github.com/vince-styling/,下面简单的说一下这个框架文件下载的实现和原理,

	// 1		RequestQueue queue = Netroid.newRequestQueue(getApplicationContext(), null);		// 2		mDownloder = new FileDownloader(queue, 1) {			@Override			public FileDownloadRequest buildRequest(String storeFilePath, String url) {				return new FileDownloadRequest(storeFilePath, url) {					@Override					public void prepare() {						addHeader("Accept-Encoding", "identity");						super.prepare();					}				};			}		};		// 3		task.controller = mDownloder.add(mSaveDirPath + task.storeFileName, task.url, new Listener<Void>() {			@Override			public void onPreExecute() {				task.invalidate();			}			@Override			public void onSuccess(Void response) {				showToast(task.storeFileName + " Success!");			}			@Override			public void onError(NetroidError error) {				NetroidLog.e(error.getMessage());			}			@Override			public void onFinish() {				NetroidLog.e("onFinish size : " + Formatter.formatFileSize(						FileDownloadActivity.this, new File(mSaveDirPath + task.storeFileName).length()));				task.invalidate();			}			@Override			public void onProgressChange(long fileSize, long downloadedSize) {				task.onProgressChange(fileSize, downloadedSize);//				NetroidLog.e("---- fileSize : " + fileSize + " downloadedSize : " + downloadedSize);			}		});

1.创建一个请求队列

2.构建一个文件下载管理器

3.将下载任务添加到队列

现在根据上面的三步来看一下它的实现原理：

第一步：创建一个请求队列:RequestQueue queue = Netroid.newRequestQueue(getApplicationContext(), null);

/**     * Creates a default instance of the worker pool and calls [email protected] RequestQueue#start()} on it.     * @param context A [email protected] Context} to use for creating the cache dir.     * @return A started [email protected] RequestQueue} instance.     */    public static RequestQueue newRequestQueue(Context context, DiskCache cache) {		int poolSize = RequestQueue.DEFAULT_NETWORK_THREAD_POOL_SIZE;		HttpStack stack;		String userAgent = "netroid/0";		try {			String packageName = context.getPackageName();			PackageInfo info = context.getPackageManager().getPackageInfo(packageName, 0);			userAgent = packageName + "/" + info.versionCode;		} catch (NameNotFoundException e) {		}		if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.GINGERBREAD) {			stack = new HurlStack(userAgent, null);		} else {			// Prior to Gingerbread, HttpUrlConnection was unreliable.			// See: http://android-developers.blogspot.com/2011/09/androids-http-clients.html			stack = new HttpClientStack(userAgent);		}		//实例化BasicNetwork，主要用于执行下载请求		Network network = new BasicNetwork(stack, HTTP.UTF_8);		//创建请求队列		RequestQueue queue = new RequestQueue(network, poolSize, cache);		//很重要的一步		queue.start();        return queue;    }

  /**     * Starts the dispatchers in this queue.     */    public void start() {        stop();  // Make sure any currently running dispatchers are stopped.        // Create the cache dispatcher and start it.        mCacheDispatcher = new CacheDispatcher(mCacheQueue, mNetworkQueue, mCache, mDelivery);        mCacheDispatcher.start();        // Create network dispatchers (and corresponding threads) up to the pool size.        for (int i = 0; i < mDispatchers.length; i++) {        	//一个线程，从请求队列中获取任务并执行            NetworkDispatcher networkDispatcher =					new NetworkDispatcher(mNetworkQueue, mNetwork, mCache, mDelivery);            mDispatchers[i] = networkDispatcher;            //Thread run()            networkDispatcher.start();        }    }    /**     * Stops the cache and network dispatchers.     */    public void stop() {        if (mCacheDispatcher != null) {            mCacheDispatcher.quit();        }		for (NetworkDispatcher mDispatcher : mDispatchers) {			//Thread interrupt()线程中断			if (mDispatcher != null) mDispatcher.quit();		}    }

框架中对于文件是没有缓存机制的，所以mCacheDispatcher可以不用理它，看一下NetworkDispatcher这个线程做了什么：com.duowan.mobile.netroid.NetworkDispatcher

public class NetworkDispatcher extends Thread {    @Override    public void run() {    	//设置线程优先级        Process.setThreadPriority(Process.THREAD_PRIORITY_BACKGROUND);        Request request;        while (true) {            try {                // Take a request from the queue.如果队列为空，则阻塞                request = mQueue.take();            } catch (InterruptedException e) {                // We may have been interrupted because it was time to quit.唯有线程中断的时候mQuit才为true,InterruptedException为中断异常            	//mQueue.take()如果队列为null,只会阻塞，不会跑出异常                if (mQuit) return;                continue;            }            try {                request.addMarker("network-queue-take");                //准备执行				mDelivery.postPreExecute(request);                // If the request was cancelled already,                // do not perform the network request.                if (request.isCanceled()) {                    request.finish("network-discard-cancelled");					mDelivery.postCancel(request);					mDelivery.postFinish(request);                    continue;                }                // Perform the network request.最重要一步！Netroid实例化的BasicNetwork在这里执行网络请求                NetworkResponse networkResponse = mNetwork.performRequest(request);                request.addMarker("network-http-complete");                // Parse the response here on the worker thread.重命名一下，没做什么                Response<?> response = request.parseNetworkResponse(networkResponse);                request.addMarker("network-parse-complete");                // Write to cache if applicable.				if (mCache != null && request.shouldCache() && response.cacheEntry != null) {					response.cacheEntry.expireTime = request.getCacheExpireTime();					mCache.putEntry(request.getCacheKey(), response.cacheEntry);					request.addMarker("network-cache-written");				}                // Post the response back.                request.markDelivered();                mDelivery.postResponse(request, response);            } catch (NetroidError netroidError) {				mDelivery.postError(request, request.parseNetworkError(netroidError));            } catch (Exception e) {				NetroidLog.e(e, "Unhandled exception %s", e.toString());				mDelivery.postError(request, new NetroidError(e));			}        }    }}

这里最重要的一步就是NetworkResponse networkResponse = mNetwork.performRequest(request);执行网络请求，但是我们不要忘记我们的mQueue还是空的，mQueue.take()正在阻塞着呢，所以，现在还没有办法进行网络请求，因此我们需要在mQueue中填充任务，才能进行我们的网络请求。不要忘记这里哦，因为我们还会回到这里！

第二步：创建一个文件下载管理器：new FileDownloader(queue, 1)

mDownloder = new FileDownloader(queue, 1) {			@Override			public FileDownloadRequest buildRequest(String storeFilePath, String url) {				return new FileDownloadRequest(storeFilePath, url) {					@Override					public void prepare() {						addHeader("Accept-Encoding", "identity");						super.prepare();					}				};			}		};

	/** The parallel task count, recommend less than 3. */	private final int mParallelTaskCount;	/** The linked Task Queue. */	private final LinkedList<DownloadController> mTaskQueue;	/**	 * Construct Downloader and init the Task Queue.	 * @param queue The RequestQueue for dispatching Download task.	 * @param parallelTaskCount	 * 				Allows parallel task count,	 * 				don't forget the value must less than ThreadPoolSize of the RequestQueue.	 */	public FileDownloader(RequestQueue queue, int parallelTaskCount) {		if (parallelTaskCount >= queue.getThreadPoolSize()) {			throw new IllegalArgumentException("parallelTaskCount[" + parallelTaskCount					+ "] must less than threadPoolSize[" + queue.getThreadPoolSize() + "] of the RequestQueue.");		}		mTaskQueue = new LinkedList<DownloadController>();		mParallelTaskCount = parallelTaskCount;		mRequestQueue = queue;	}

第三步：将下载任务添加到队列，task.controller = mDownloder.add(mSaveDirPath + task.storeFileName, task.url, new Listener<Void>()：

	/**	 * Create a new download request, this request might not run immediately because the parallel task limitation,	 * you can check the status by the [email protected] DownloadController} which you got after invoke this method.	 *	 * Note: don't perform this method twice or more with same parameters, because we didn't check for	 * duplicate tasks, it rely on developer done.	 *	 * Note: this method should invoke in the main thread.	 *	 * @param storeFilePath Once download successed, we'll find it by the store file path.	 * @param url The download url.	 * @param listener The event callback by status;	 * @return The task controller allows pause or resume or discard operation.	 */	public DownloadController add(String storeFilePath, String url, Listener<Void> listener) {		// only fulfill requests that were initiated from the main thread.(reason for the Delivery?)		//看名字就知道		throwIfNotOnMainThread();		//创建一个下载控制器		DownloadController controller = new DownloadController(storeFilePath, url, listener);		synchronized (mTaskQueue) {			//这可不是mQueue,这里只是一个DownloadController的LinkedList集合			mTaskQueue.add(controller);		}		//重点来了		schedule();		return controller;	}

	/**	 * Traverse the Task Queue, count the running task then deploy more if it can be.	 */	private void schedule() {		// make sure only one thread can manipulate the Task Queue.		synchronized (mTaskQueue) {			// counting ran task.			int parallelTaskCount = 0;			for (DownloadController controller : mTaskQueue) {				//累计队列中正在下载的的任务数				if (controller.isDownloading()) parallelTaskCount++;			}			//当正在下载的个数大于并行任务数的时候，不在执行下载任务			/*			 * 这里举个例子说明一下：我们默认mParallelTaskCount=1			 * 当我们添加第一个任务的时候，这个的controller.isDownloading()肯定是false			 * 所以parallelTaskCount >= mParallelTaskCount是不成立的，当我们再添加一个任务的时候，现在mTaskQueue.size是2了			 * 且第一个isDownloading，为了保证并发数量为1，会return,说的有点乱，不知道说明白了没有			 */			if (parallelTaskCount >= mParallelTaskCount) return;			// try to deploy all Task if they're await.			for (DownloadController controller : mTaskQueue) {				//deploy()，将任务添加到队列中				if (controller.deploy() && ++parallelTaskCount == mParallelTaskCount) return;			}		}	}

		/**		 * For the parallel reason, only the [email protected] FileDownloader#schedule()} can call this method.		 * @return true if deploy is successed.		 */		private boolean deploy() {			if (mStatus != STATUS_WAITING) return false;			//第二步我说很吓人那个地方			mRequest = buildRequest(mStoreFilePath, mUrl);			// we create a Listener to wrapping that Listener which developer specified,			// for the onFinish(), onSuccess(), onError() won't call when request was cancel reason.			mRequest.setListener(new Listener<Void>() {				boolean isCanceled;				@Override				public void onPreExecute() {					mListener.onPreExecute();				}				@Override				public void onFinish() {					// we don't inform FINISH when it was cancel.					if (!isCanceled) {						mStatus = STATUS_PAUSE;						mListener.onFinish();						// when request was FINISH, remove the task and re-schedule Task Queue.//						remove(DownloadController.this);					}				}				@Override				public void onSuccess(Void response) {					// we don't inform SUCCESS when it was cancel.					if (!isCanceled) {						mListener.onSuccess(response);						mStatus = STATUS_SUCCESS;						remove(DownloadController.this);					}				}				@Override				public void onError(NetroidError error) {					// we don't inform ERROR when it was cancel.					if (!isCanceled) mListener.onError(error);				}				@Override				public void onCancel() {					mListener.onCancel();					isCanceled = true;				}				@Override				public void onProgressChange(long fileSize, long downloadedSize) {					mListener.onProgressChange(fileSize, downloadedSize);				}			});			mStatus = STATUS_DOWNLOADING;			//我擦，终于把任务加到队列中了			mRequestQueue.add(mRequest);			return true;		}

 public Request add(Request request) {        // Tag the request as belonging to this queue and add it to the set of current requests.        request.setRequestQueue(this);        synchronized (mCurrentRequests) {            mCurrentRequests.add(request);        }        // Process requests in the order they are added.        request.setSequence(getSequenceNumber());        request.addMarker("add-to-queue");        // If the request is uncacheable or forceUpdate, skip the cache queue and go straight to the network.        if (request.isForceUpdate() || !request.shouldCache()) {			mDelivery.postNetworking(request);			mNetworkQueue.add(request);			return request;        }}

request.shouldCache()有兴趣的可以自己去看一下，这里说明了文件下载没有缓存机制，这里就不多说了，因为如果你还没有忘记的话，mQueue.take()还在阻塞着呢，好了让我们回到第一步，执行网络请求

 NetworkResponse networkResponse = mNetwork.performRequest(request);

	@Override	public NetworkResponse performRequest(Request<?> request) throws NetroidError {		// Determine if request had non-http perform.		NetworkResponse networkResponse = request.perform();		if (networkResponse != null) return networkResponse;		long requestStart = SystemClock.elapsedRealtime();		while (true) {			// If the request was cancelled already,			// do not perform the network request.			if (request.isCanceled()) {				request.finish("perform-discard-cancelled");				mDelivery.postCancel(request);				throw new NetworkError(networkResponse);			}			HttpResponse httpResponse = null;			byte[] responseContents = null;			try {				// prepare to perform this request, normally is reset the request headers.				request.prepare();				httpResponse = mHttpStack.performRequest(request);				StatusLine statusLine = httpResponse.getStatusLine();				int statusCode = statusLine.getStatusCode();				responseContents = request.handleResponse(httpResponse, mDelivery);				if (statusCode < 200 || statusCode > 299) throw new IOException();				// if the request is slow, log it.				long requestLifetime = SystemClock.elapsedRealtime() - requestStart;				logSlowRequests(requestLifetime, request, responseContents, statusLine);				return new NetworkResponse(statusCode, responseContents, parseCharset(httpResponse));			} catch (SocketTimeoutException e) {				attemptRetryOnException("socket", request, new TimeoutError());			} catch (ConnectTimeoutException e) {				attemptRetryOnException("connection", request, new TimeoutError());			} catch (MalformedURLException e) {				throw new RuntimeException("Bad URL " + request.getUrl(), e);			} catch (IOException e) {				if (httpResponse == null) throw new NoConnectionError(e);				int statusCode = httpResponse.getStatusLine().getStatusCode();				NetroidLog.e("Unexpected response code %d for %s", statusCode, request.getUrl());				if (responseContents != null) {					networkResponse = new NetworkResponse(statusCode, responseContents, parseCharset(httpResponse));					if (statusCode == HttpStatus.SC_UNAUTHORIZED || statusCode == HttpStatus.SC_FORBIDDEN) {						attemptRetryOnException("auth", request, new AuthFailureError(networkResponse));					} else {						// TODO: Only throw ServerError for 5xx status codes.						throw new ServerError(networkResponse);					}				} else {					throw new NetworkError(networkResponse);				}			}		}	}

responseContents = request.handleResponse(httpResponse, mDelivery);，写文件,断点续传的原理

	/**	 * In this method, we got the Content-Length, with the TemporaryFile length,	 * we can calculate the actually size of the whole file, if TemporaryFile not exists,	 * we'll take the store file length then compare to actually size, and if equals,	 * we consider this download was already done.	 * We used [email protected] RandomAccessFile} to continue download, when download success,	 * the TemporaryFile will be rename to StoreFile.	 */	@Override	public byte[] handleResponse(HttpResponse response, Delivery delivery) throws IOException, ServerError {		// Content-Length might be negative when use HttpURLConnection because it default header Accept-Encoding is gzip,		// we can force set the Accept-Encoding as identity in prepare() method to slove this problem but also disable gzip response.		HttpEntity entity = response.getEntity();		//获取文件的总大小		long fileSize = entity.getContentLength();		if (fileSize <= 0) {			NetroidLog.d("Response doesn't present Content-Length!");		}				long downloadedSize = mTemporaryFile.length();		/*		 * 是否支持断点续传		 * 		 * 客户端每次提交下载请求时，服务端都要添加这两个响应头，以保证客户端和服务端将此下载识别为可以断点续传的下载：		 *  Accept-Ranges：告知下载客户端这是一个可以恢复续传的下载，存放本次下载的开始字节位置、文件的字节大小；		 *  ETag：保存文件的唯一标识（我在用的文件名+文件最后修改时间，以便续传请求时对文件进行验证）；		 *  Last-Modified：可选响应头，存放服务端文件的最后修改时间，用于验证		 */		boolean isSupportRange = HttpUtils.isSupportRange(response);		if (isSupportRange) {			fileSize += downloadedSize;			// Verify the Content-Range Header, to ensure temporary file is part of the whole file.			// Sometime, temporary file length add response content-length might greater than actual file length,			// in this situation, we consider the temporary file is invalid, then throw an exception.			String realRangeValue = HttpUtils.getHeader(response, "Content-Range");			// response Content-Range may be null when "Range=bytes=0-"			if (!TextUtils.isEmpty(realRangeValue)) {				String assumeRangeValue = "bytes " + downloadedSize + "-" + (fileSize - 1);				if (TextUtils.indexOf(realRangeValue, assumeRangeValue) == -1) {					throw new IllegalStateException(							"The Content-Range Header is invalid Assume[" + assumeRangeValue + "] vs Real[" + realRangeValue + "], " +									"please remove the temporary file [" + mTemporaryFile + "].");				}			}		}		// Compare the store file size(after download successes have) to server-side Content-Length.		// temporary file will rename to store file after download success, so we compare the		// Content-Length to ensure this request already download or not.		if (fileSize > 0 && mStoreFile.length() == fileSize) {			// Rename the store file to temporary file, mock the download success. ^_^			mStoreFile.renameTo(mTemporaryFile);			// Deliver download progress.			delivery.postDownloadProgress(this, fileSize, fileSize);			return null;		}		//之所以能够实现断点续传的原因所在		RandomAccessFile tmpFileRaf = new RandomAccessFile(mTemporaryFile, "rw");		// If server-side support range download, we seek to last point of the temporary file.		if (isSupportRange) {			//移动文件读写指针位置			tmpFileRaf.seek(downloadedSize);		} else {			// If not, truncate the temporary file then start download from beginning.			tmpFileRaf.setLength(0);			downloadedSize = 0;		}		try {			InputStream in = entity.getContent();			// Determine the response gzip encoding, support for HttpClientStack download.			if (HttpUtils.isGzipContent(response) && !(in instanceof GZIPInputStream)) {				in = new GZIPInputStream(in);			}			byte[] buffer = new byte[6 * 1024]; // 6K buffer			int offset;			while ((offset = in.read(buffer)) != -1) {				//写文件				tmpFileRaf.write(buffer, 0, offset);				downloadedSize += offset;				long currTime = SystemClock.uptimeMillis();				//控制下载进度的速度				if (currTime - lastUpdateTime >= DEFAULT_TIME) {					lastUpdateTime = currTime;					delivery.postDownloadProgress(this, fileSize,							downloadedSize);				}				if (isCanceled()) {					delivery.postCancel(this);					break;				}			}		} finally {			try {				// Close the InputStream and release the resources by "consuming the content".				if (entity != null) entity.consumeContent();			} catch (Exception e) {				// This can happen if there was an exception above that left the entity in				// an invalid state.				NetroidLog.v("Error occured when calling consumingContent");			}			tmpFileRaf.close();		}		return null;	}

好了，Netroid的原理基本上就是这些了，讲一下我用的时候遇到的两个问题：

1.下载进度的速度太快，你如果用notifition来显示，会出现ANR,所以我们要控制一下它的速度，具体方法在上面

//控制下载进度的速度				if (currTime - lastUpdateTime >= DEFAULT_TIME) {					lastUpdateTime = currTime;					delivery.postDownloadProgress(this, fileSize,							downloadedSize);				}

	@Override				public void onFinish() {					// we don't inform FINISH when it was cancel.					if (!isCanceled) {						mStatus = STATUS_PAUSE;						mListener.onFinish();						// when request was FINISH, remove the task and re-schedule Task Queue.//						remove(DownloadController.this);					}				}				@Override				public void onSuccess(Void response) {					// we don't inform SUCCESS when it was cancel.					if (!isCanceled) {						mListener.onSuccess(response);						mStatus = STATUS_SUCCESS;						remove(DownloadController.this);					}				}

把onFinish的status改成STATUS_PAUSE，并去掉remove(DownloadController.this);，在onSuccess中再将status修改为STATUS_SUCCESS，并remove，当然这个办法治标不治本，如果有谁知道请告之，谢谢！