[Java]_[中级]_[使用okhttp3和HttpClient代理访问外部网络]

场景

Java的http库常用的有HttpClient和Okhttp3, 如果公司有限制网络访问，需要代理才可以访问外网，那么如何使用代理Proxy？

<dependency>
	<groupId>org.apache.httpcomponents</groupId>
	<artifactId>httpclient</artifactId>
	<version>4.5.13</version>
</dependency>

<dependency>
	<groupId>com.squareup.okhttp3</groupId>
	<artifactId>okhttp</artifactId>
	<version>3.14.9</version>
</dependency>

说明

从性能方面来说，Okhttp3更出色一点。因为从我本机运行两个库的访问, Okhttp3会快一些。
从易用性来说,Ohttp3更好。OkHttpClient只需要在Builder里调用proxy(Proxy)传递一个Proxy对象之后即可。之后访问其他外网，就直接创建外网的Request即可，非常方便。代理和非代理都用统一的方法。
```
OkHttpClient.Builder builder = client.newBuilder();
builder.proxy(new Proxy(Proxy.Type.HTTP, 
	new InetSocketAddress(hostName, Integer.parseInt(hostPort))));

...
Response response = getClient().newCall(request).execute()
```

HttpClient访问^[1]则需要创建两个对象: 需要访问的主机HttpHost和URL里的请求路径和查询参数的HttpGet对象。每次请求都需要传递这两个参数。而如果非代理的话，就需要调用另外的重载方法。

 HttpHost target = HttpHost.create(hostUrl);
HttpGet request = new HttpGet("/user?name=xxx");
//  HttpGet httpGet = new HttpGet(hostUrl); // 非代理

...
httpClient.execute(target, request) // 代理
// httpClient.execute(httpGet); // 非代理

注意，出于创建Client比较废资源，okhttp3还是HttpClient官方都是推荐创建单例的OkHttpClient和HttpClient对象来使用。对于CloseableHttpClient在使用完之后还需要调用close()关闭。OkHttpClient官方例子也没有调用销毁方法。
从源码设计上来说，Okhttp3在发送数据时，使用了自带的线程池，我们想用自己的线程池代替都不行。这个对某些线程数有严格限制的程序(Linux)非常不友好，甚至不能用，因为它的线程不受宿主程序控制。
```
// 在类 RealConnectionPool里有这段代码
static {
    executor = new ThreadPoolExecutor(0, 2147483647, 60L, TimeUnit.SECONDS, new SynchronousQueue(), Util.threadFactory("OkHttp ConnectionPool", true));
}
```

例子

以下是使用okhttp3和HttpClient的目前最佳做法.

OkHttpClientCaller


import Pair;
import StringUtils;
import HttpHelper;
import okhttp3.*;
import org.apache.log4j.Logger;

import java.io.IOException;
import java.net.InetSocketAddress;
import java.net.Proxy;
import java.util.Map;

public class OkHttpClientCaller {

    private static Logger logger = Logger.getLogger(OkHttpClientCaller.class);

    private static OkHttpClient client;

    public static void newClient(){
        OkHttpClient.Builder builder = new OkHttpClient().newBuilder();
        client = builder.cookieJar(new LocalCookieJar()).build();
    }

    public static void setProxy(String hostName,String hostPort){

        Proxy proxy = client.proxy();
        if(StringUtils.isEmpty(hostName,hostPort)){
            if(proxy != null) {
                client.newBuilder().proxy(null).cookieJar(new LocalCookieJar()).build();
            }
        }else{
            if(proxy == null){
                OkHttpClient.Builder builder = client.newBuilder();
                builder.proxy(new Proxy(Proxy.Type.HTTP, new InetSocketAddress(hostName, Integer.parseInt(hostPort))));
                client = builder.cookieJar(new LocalCookieJar()).build();
            }
        }

    }

    public static OkHttpClient getClient(){
        return client;
    }

    public static Pair<Boolean,String> doGet(Map<String,String> hostHeaders,String hostUrl){
        Request.Builder builder = new Request.Builder().url(hostUrl);
        HttpHelper.buildRequestHeader(builder,hostHeaders);
        Request request = builder.get().build();

        try (Response response = getClient().newCall(request).execute()) {
            return new Pair<>((response.code() == 200), response.body().string());
        } catch (Exception e) {
            logger.error(e.getMessage());
        }

        return new Pair<>(false,"");
    }

    /**
     *
     * @param apiUrl
     * @param apiHeaders
     * @param contentType "application/x-www-form-urlencoded" "application/json"
     * @param content
     * @return
     */
    public static Pair<Boolean,String> doPost(String apiUrl,
                                              Map<String,String> apiHeaders,
                                              String contentType,String content){
        // Header
        Request.Builder builder = new Request.Builder().url(apiUrl);
        HttpHelper.buildRequestHeader(builder,apiHeaders);

        MediaType mediaType = MediaType.parse(contentType);
        RequestBody body = RequestBody.create(mediaType,content);

        // Post
        try(Response response = getClient().newCall(builder.post(body).build()).execute()){
            return new Pair<>((response.code() == 200), response.body().string());
        } catch (IOException e) {
            logger.error(e.getMessage());
        }

        return new Pair<>(false,"");
    }

    public static Pair<Boolean,String> doPostFormData(String apiUrl,
                                              Map<String,String> apiHeaders,
                                              String content){
        return doPost(apiUrl,apiHeaders,"application/x-www-form-urlencoded",content);
    }

    public static Pair<Boolean,String> doPostJsonData(String apiUrl,
                                                      Map<String,String> apiHeaders,
                                                      String content){
        return doPost(apiUrl,apiHeaders,"application/json",content);
    }
}

HttpClientCaller


import Pair;
import StringUtils;
import RequestUtils;
import HttpHelper;
import org.apache.http.HttpHost;
import org.apache.http.client.HttpRequestRetryHandler;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.conn.ssl.NoopHostnameVerifier;
import org.apache.http.conn.ssl.SSLConnectionSocketFactory;
import org.apache.http.conn.ssl.TrustStrategy;
import org.apache.http.entity.StringEntity;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.protocol.HttpContext;
import org.apache.http.ssl.SSLContextBuilder;
import org.apache.http.util.EntityUtils;
import org.apache.log4j.Logger;

import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.SSLContext;
import java.io.IOException;
import java.net.URL;
import java.security.KeyManagementException;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;
import java.util.Map;

import static RequestUtils.getPathAndQueryParam;
import static RequestUtils.getPort;

public class HttpClientCaller {

    private static Logger logger = Logger.getLogger(HttpClientCaller.class);

    private static CloseableHttpClient httpClient;
    private static RequestConfig requestConfig;

    //传输超时时间, 10秒
    private static int socketTimeout = 10000;

    //建立连接超时时间，默认10秒
    private static int connectTimeout = 5000;

    //获取连接超时时间，10秒
    private static int connectRequest = 5000;

    private static HttpHost proxy = null;

    private static HttpRequestRetryHandler myRetryHandler = new HttpRequestRetryHandler() {
        @Override
        public boolean retryRequest(IOException exception,
                                    int executionCount, HttpContext context) {
            return false;
        }
    };

    public static CloseableHttpClient createClientAttr(HttpClientBuilder builder){
        return builder.setRetryHandler(myRetryHandler).setMaxConnPerRoute(2).setMaxConnTotal(2).build();
    }

    public static void newClient() {

        // setMaxConnTotal是连接池中可用的总最大连接数。setMaxConnPerRoute是限制到单个端口或url的连接总数。
        httpClient = createClientAttr(HttpClients.custom());

        //根据默认超时限制初始化requestConfig
        requestConfig = getCommonRequestConfigBuilder().build();

    }

    public static RequestConfig.Builder getCommonRequestConfigBuilder(){

        RequestConfig.Builder builder = RequestConfig.custom().setConnectionRequestTimeout(connectRequest)
                .setSocketTimeout(socketTimeout)
                .setConnectTimeout(connectTimeout);
        return builder;
    }

    /**
     * 不行: https://blog.csdn.net/woshirongshaolin/article/details/126992654
     * @return
     */
    public static SSLConnectionSocketFactory createSSLFactory(){

        //使用 loadTrustMaterial() 方法实现一个信任策略，信任所有证书
        try {
            SSLContext sslContext = new SSLContextBuilder().loadTrustMaterial(null, new TrustStrategy() {
                // 信任所有
                public boolean isTrusted(X509Certificate[] chain, String authType) throws CertificateException {
                    return true;
                }
            }).build();

            //NoopHostnameVerifier类:  作为主机名验证工具，实质上关闭了主机名验证，它接受任何
            //有效的SSL会话并匹配到目标主机。
            HostnameVerifier hostnameVerifier = NoopHostnameVerifier.INSTANCE;
            SSLConnectionSocketFactory sslsf = new SSLConnectionSocketFactory(sslContext, hostnameVerifier);
            return sslsf;
        } catch (NoSuchAlgorithmException e) {
            logger.error(e.getMessage());
        } catch (KeyManagementException e) {
            logger.error(e.getMessage());
        } catch (KeyStoreException e) {
            logger.error(e.getMessage());
        }

        return null;
    }

    /**
     *
     * @param hostName
     * @param port
     */
    public static void setProxy(String hostName,String port){

        // 不支持
        // https://github.com/apache/httpcomponents-client/blob/4.5.x/httpclient/src/examples/org/apache/http/examples/client/ClientExecuteProxy.java
        if(StringUtils.isEmpty(hostName,port)){
            if(proxy != null)
                requestConfig = getCommonRequestConfigBuilder().setProxy(null).build();
        }else{
            if(proxy == null){
                closeClient();
                SSLConnectionSocketFactory sslFactory = createSSLFactory();
                if(sslFactory != null)
                    httpClient = createClientAttr(HttpClients.custom().setSSLSocketFactory(sslFactory));

                proxy = new HttpHost(hostName, Integer.parseInt(port));
                requestConfig = getCommonRequestConfigBuilder().setProxy(proxy).build();
            }
        }
    }

    public static void closeClient(){
        try {
            if(httpClient != null)
                httpClient.close();

            httpClient = null;
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    public static RequestConfig getRequestConfig(){
        return requestConfig;
    }

    public static CloseableHttpClient getClient(){
        return httpClient;
    }

    public static Pair<Boolean,String> doGet(Map<String,String> hostHeaders, String hostUrl){

        if(proxy != null){

            URL url = RequestUtils.getUrlOrNull(hostUrl);

            HttpHost target = new HttpHost(url.getHost(), getPort(url),url.getProtocol());
            HttpGet request = new HttpGet(getPathAndQueryParam(url));
            request.setConfig(requestConfig);

            HttpHelper.buildRequestHeader(request,hostHeaders);
            try(CloseableHttpResponse response = httpClient.execute(target, request);){
                return new Pair<>((response.getStatusLine().getStatusCode() == 200),
                        EntityUtils.toString(response.getEntity(), "utf-8"));
            }catch ( IOException e){
                e.printStackTrace();
            }

        }else{
            HttpGet httpGet = new HttpGet(hostUrl);
            httpGet.setConfig(requestConfig);

            HttpHelper.buildRequestHeader(httpGet,hostHeaders);
            try(CloseableHttpResponse response = httpClient.execute(httpGet);){
                return new Pair<>((response.getStatusLine().getStatusCode() == 200),
                        EntityUtils.toString(response.getEntity(), "utf-8"));
            }catch ( IOException e){
                e.printStackTrace();
            }
        }

        return new Pair<>(false,"");
    }

    public static Pair<Boolean,String> doPost(String apiUrl,
                                              Map<String,String> apiHeaders,
                                              String contentType,String content) {
        if(proxy != null){

            URL url = RequestUtils.getUrlOrNull(apiUrl);
            HttpHost target = new HttpHost(url.getHost(), getPort(url),url.getProtocol());
            HttpPost post = new HttpPost(getPathAndQueryParam(url));

            HttpHelper.buildRequestHeader(post,apiHeaders);
            if(post.getFirstHeader("Content-Type") == null)
                post.addHeader("Content-Type",contentType);

            StringEntity postEntity = new StringEntity(content, "UTF-8");
            post.setEntity(postEntity);
            post.setConfig(requestConfig);
            try(CloseableHttpResponse response = httpClient.execute(target,post)){
                return new Pair<>((response.getStatusLine().getStatusCode() == 200),
                        EntityUtils.toString(response.getEntity(), "utf-8"));
            } catch (IOException e) {
                logger.warn(e.getMessage());
            }

        }else{
            HttpPost post = new HttpPost(apiUrl);
            HttpHelper.buildRequestHeader(post,apiHeaders);
            if(post.getFirstHeader("Content-Type") == null)
                post.addHeader("Content-Type",contentType);

            StringEntity postEntity = new StringEntity(content, "UTF-8");
            post.setEntity(postEntity);
            post.setConfig(requestConfig);
            try(CloseableHttpResponse response = httpClient.execute(post)){
                return new Pair<>((response.getStatusLine().getStatusCode() == 200),
                        EntityUtils.toString(response.getEntity(), "utf-8"));
            } catch (IOException e) {
                logger.warn(e.getMessage());
            }
        }


        return  new Pair<>(false,"");
    }

    public static Pair<Boolean,String> doPostFormData(String apiUrl,
                                                      Map<String,String> apiHeaders,
                                                      String content){
        return doPost(apiUrl,apiHeaders,"application/x-www-form-urlencoded; charset=UTF-8",content);
    }

    public static Pair<Boolean,String> doPostJsonData(String apiUrl,
                                                      Map<String,String> apiHeaders,
                                                      String content){
        return doPost(apiUrl,apiHeaders,"application/json; charset=UTF-8",content);
    }
}

HttpHelper


import okhttp3.Request;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.message.AbstractHttpMessage;

import java.util.Map;

public class HttpHelper {

    public static void buildRequestHeader(AbstractHttpMessage builder, Map<String,String> values){
        for (Map.Entry<String,String> keyValue:values.entrySet()) {
            builder.addHeader(keyValue.getKey(),keyValue.getValue());
        }
    }

    public static void buildRequestHeader(Request.Builder builder,Map<String,String> values){
        for (Map.Entry<String,String> keyValue:values.entrySet()) {
            builder.addHeader(keyValue.getKey(),keyValue.getValue());
        }
    }

    public static String toFormData(Map<String,String> params){
        StringBuilder sb = new StringBuilder();
        for (Map.Entry<String,String> keyValue: params.entrySet())
            sb.append(keyValue.getKey()).append("=").append(keyValue.getValue()).append("&");

        if(sb.length() > 0)
            return sb.substring(0,sb.length()-1);

        return "";
    }
}

RequestUtils

import java.net.MalformedURLException;
import java.net.URL;

public class RequestUtils {

    /**
     *
     * @param url
     * @return URL or null
     */
    public static URL getUrlOrNull(String url){
        try {
            return new URL(url);
        } catch (MalformedURLException e) {
            e.printStackTrace();
        }

        return null;
    }

    public static int getPort(URL url){
        int port = url.getPort();
        return (port == -1)?url.getDefaultPort():port;
    }

    public static String getPathAndQueryParam(URL url){
        String path = url.getPath();
        String query = url.getQuery();
        if(query == null)
            return path;

        return String.format("%s?%s",path,query);
    }
}

运行

OkHttpClientCaller.newClient();
OkHttpClientCaller.setProxy("127.0.0.1","1283");
Pair<Boolean, String> pp = OkHttpClientCaller.doGet(headers, "https://www.bing.com");
if(pp.k)
    logger.info(pp.v);

OkHttpClientCaller.closeClient();

参考

httpcomponents-client/httpclient/src/examples/org/apache/http/examples/client/ClientExecuteProxy.java
Httpclient3.1指定tlsv1.2协议访问https
java http请求设置代理 Proxy_java
RestTemplate发起HTTPS请求Unsupported or unrecognized SSL message
HttpClient访问不安全的https链接报错:SSLHandshakeException
httpclient信任所有证书解决SSLException:Unrecognized SSL message
Java URI getQuery()方法及示例文章来源地址https://uudwc.com/A/wozd8