2020年3月9日

1、邀测的接口，不能直接在控制台调用，可通过百度人体分析官方QQ群（860337848）联系群管、或提交工单申请开通测试权限。

2、邀测的接口，暂未封装服务端SDK，只能通过API调用。

官方文档一定要看哦

https://ai.baidu.com/ai-doc/BODY/Jk7ir38ut

接口描述

对于输入的一张图片（可正常解码，且长宽比适宜），检测图片中的手部，定位食指指尖、及4个辅助关键点的坐标位置，

模型针对儿童学习机点读场景进行专项调优，可用于点读搜题、自动跟读等应用。

当前接口主要适用于图片中单个手部的情况，图片中同时存在多个手部时，识别效果可能欠佳。

接下来就用Java调用接口 亲自体验一遍

import cn.hutool.http.HttpUtil;
import cn.xsshome.taip.util.FileUtil;
import com.alibaba.fastjson.JSON;
import com.baidu.aip.util.Base64Util;
import javax.imageio.ImageIO;
import java.awt.*;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.FileOutputStream;
import java.net.URLEncoder;

/**
 * @author 小帅丶
 * @className FingerTipSample
 * @Description 指尖检测
 * @Date 2020/3/9-9:40
 **/
public class FingerTipSample {
    //指尖识别接口
    public static String REFUSE_URL = "https://aip.baidubce.com/rest/2.0/image-classify/v1/fingertip";

    public static void main(String[] args) throws Exception{
        String access_token = "";
        //要检测的图片
        String filePath = "F://testimg//fingertip3.jpg";
        //图片转base64字符串处理
        String base64Img = Base64Util.encode(FileUtil.readFileByBytes(filePath));
        //参数拼接
        String param = "image="+ URLEncoder.encode(base64Img, "UTF-8");
        //发送请求并获取结果
        String result = HttpUtil.post(REFUSE_URL+"?access_token="+access_token, param);
        System.out.println(result);
        FingerTipBean fingerTipBean = JSON.parseObject(result, FingerTipBean.class);
        getAfterImage(filePath,fingerTipBean);

    }
    /**
     * @Author 小帅丶
     * @Description 给图片增加框
     * @Date  2020/2/27 13:11
     * @param filePath 图片路径
     * @param bean 包含坐标的对象 x-left y-top width-width height-height
     * @return void
     **/
    public static void getAfterImage(String filePath,FingerTipBean bean) throws Exception{
        String outputfilePath = "F://testimg//fingertip//afterfingertip.jpg";
        BufferedImage image = ImageIO.read(new File(filePath));
        Graphics g = image.getGraphics();
        Integer dotSize = 8;
        if(bean.getHand_num()>0){
            //指尖所在图片的位置
            g.drawString("识别出指尖:"+ bean.getHand_num() + "个",100,100);
            g.setColor(Color.BLACK);
            g.drawRect(bean.getHand_info().get(0).getLocation().getLeft(), bean.getHand_info().get(0).getLocation().getTop(), bean.getHand_info().get(0).getLocation().getWidth(),bean.getHand_info().get(0).getLocation().getHeight());

            g.setColor(Color.BLUE);
            g.drawString("食指", bean.getHand_info().get(0).getHand_parts().getIndexFinger().getX(),bean.getHand_info().get(0).getHand_parts().getIndexFinger().getY());
            g.fillOval(bean.getHand_info().get(0).getHand_parts().getIndexFinger().getX(),bean.getHand_info().get(0).getHand_parts().getIndexFinger().getY(),dotSize,dotSize);

            g.setColor(Color.RED);
            g.drawString("食指中间关节", bean.getHand_info().get(0).getHand_parts().getIndexFingerJoint().getX(),bean.getHand_info().get(0).getHand_parts().getIndexFingerJoint().getY());
            g.fillOval(bean.getHand_info().get(0).getHand_parts().getIndexFingerJoint().getX(),bean.getHand_info().get(0).getHand_parts().getIndexFingerJoint().getY(),dotSize,dotSize);

            g.setColor(Color.GREEN);
            g.drawString("食指指根", bean.getHand_info().get(0).getHand_parts().getIndexFingerRoot().getX(),bean.getHand_info().get(0).getHand_parts().getIndexFingerRoot().getY());
            g.fillOval(bean.getHand_info().get(0).getHand_parts().getIndexFingerRoot().getX(),bean.getHand_info().get(0).getHand_parts().getIndexFingerRoot().getY(),dotSize,dotSize);

            g.setColor(Color.MAGENTA);
            g.drawString("中指中间关节",bean.getHand_info().get(0).getHand_parts().getMiddleFinger().getX(),bean.getHand_info().get(0).getHand_parts().getMiddleFinger().getY());
            g.fillOval(bean.getHand_info().get(0).getHand_parts().getMiddleFinger().getX(),bean.getHand_info().get(0).getHand_parts().getMiddleFinger().getY(),dotSize,dotSize);

            g.setColor(Color.ORANGE);
            g.drawString("中指指根", bean.getHand_info().get(0).getHand_parts().getMiddleFingerRoot().getX(),bean.getHand_info().get(0).getHand_parts().getMiddleFingerRoot().getY());
            g.fillOval(bean.getHand_info().get(0).getHand_parts().getMiddleFingerRoot().getX(),bean.getHand_info().get(0).getHand_parts().getMiddleFingerRoot().getY(),dotSize,dotSize);

            FileOutputStream out = new FileOutputStream(outputfilePath);//输出图片的地
            ImageIO.write(image, "jpeg", out);
            new JFrameSample(outputfilePath);
        } else{
            g.drawString("未识别出指尖",100,100);
            FileOutputStream out = new FileOutputStream(outputfilePath);//输出图片的地
            ImageIO.write(image, "jpeg", out);
            new JFrameSample(outputfilePath);
        }

    }
}

涉及到的2个工具类一个是接口返回的JSON序列化成Java对象。另一个是JFrame显示图片

需要使用lombok插件和jar

package cn;

import com.alibaba.fastjson.annotation.JSONField;
import lombok.Data;
import lombok.NoArgsConstructor;

import java.util.List;

/**
 * @author 小帅丶
 * @className FingerTipBean
 * @Description 指尖检测对象
 * @Date 2020/3/9-10:36
 **/
@NoArgsConstructor
@Data
public class FingerTipBean {

    private long log_id;
    private int hand_num;
    private List hand_info;

    @NoArgsConstructor
    @Data
    public static class HandInfoBean {
        private HandPartsBean hand_parts;
        private LocationBean location;

        @NoArgsConstructor
        @Data
        public static class HandPartsBean {
            //食指
            @JSONField(name="0")
            private IndexFinger indexFinger;
            //食指中间关节
            @JSONField(name="1")
            private IndexFingerJoint indexFingerJoint;
            //食指指根
            @JSONField(name="2")
            private IndexFingerRoot indexFingerRoot;
            //中指中间关节
            @JSONField(name="3")
            private MiddleFinger middleFinger;
            //中指指根
            @JSONField(name="4")
            private MiddleFingerRoot middleFingerRoot;

            @NoArgsConstructor
            @Data
            public static class IndexFingerJoint {
                private int y;
                private int x;
                private double score;
            }

            @NoArgsConstructor
            @Data
            public static class IndexFinger {
                private int y;
                private int x;
                private double score;
            }

            @NoArgsConstructor
            @Data
            public static class IndexFingerRoot {
                private int y;
                private int x;
                private double score;
            }

            @NoArgsConstructor
            @Data
            public static class MiddleFinger {
                private int y;
                private int x;
                private double score;
            }

            @NoArgsConstructor
            @Data
            public static class MiddleFingerRoot {
                private int y;
                private int x;
                private double score;
            }
        }

        @NoArgsConstructor
        @Data
        public static class LocationBean {
            private int height;
            private int width;
            private int top;
            private double score;
            private int left;
        }
    }
}

图片展示工具类

package cn;

import javax.swing.*;
import java.awt.*;

/**
 * @author 小帅丶
 * @className JFrameSample
 * @Description //TODO
 * @Date 2020/3/9-9:46
 **/
public class JFrameSample extends JFrame{
    /**
     * @Author 小帅丶
     * @Description 显示图片
     * @Date  2020/3/9 10:21
     * @param filePath 图片路径
     * @return
     **/
    public JFrameSample(String filePath){
        setTitle("处理后的图片");
        ImageIcon imageIcon = new ImageIcon(filePath);
        setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
        setLocationRelativeTo(null);
        setSize(imageIcon.getIconWidth(),imageIcon.getIconHeight());
        setResizable(false);
        getContentPane().setLayout(null);
        JPanel panel = new ImagePanel(filePath,imageIcon);
        panel.setBounds(0,0,imageIcon.getIconWidth(),imageIcon.getIconHeight());
        getContentPane().add(panel);
        setVisible(true);
    }
    class ImagePanel extends JPanel{
        String filePathNew = null;
        ImageIcon imageIconNew = null;
        public ImagePanel(String filePath,ImageIcon imageIcon) {
            filePathNew = filePath;
            imageIconNew = imageIcon;
        }
        @Override
        public void paint(Graphics g) {
            super.paint(g);
            g.drawImage(imageIconNew.getImage(), 0, 0, imageIconNew.getIconWidth(),imageIconNew.getIconHeight(),this);
        }
    }
}

正常识别

原图

处理后的图

非正常识别

接口初体验建议

1.接口返回的定义名称以数字命名不算特别友好

2.接口并不会告知是左手还是右手 毕竟有左撇子是吧。哈哈

3.请求参数只有一个image 后续是否支持对左右手、区域进行框架识别，接口没说明识别图片的一些要求。导致很多图片都无法识别出指尖信息。

4.整体效果并不是特别好。部分指尖关节位置有偏差

5.应用场景不是特别亲民 哈哈 我这边没想到实际的落地场景

运行的截图

个人主页：https://ai.baidu.com/forum/user/center/2312713

个人邮箱：youngxiaoshuai#163.com （#替换为@）