Deep Learning from AI to True-AI

LearningandVisionResearchGroup,NUS(NUS-LV)DeepLearning:[e-mail]ngaporeAIInstitute,Qihoo/360(soon)AI:ThreeLevelsfromBrain-inspiredPerspectiveNeuronHumanBrainBabyLearningSignalsdirectlyconvertedintonewsignal/informationSignalshierarchicallyconvertedintofinalinformationHowhumanself-learnbyinteractingwithrealworldinthewholelifeChappieAIvs.True-AIAI:Brain-likeTrue-AI:Baby-likeLearnfixedmodelsbyannotateddata1.Speechrecognition2.Facerecognition3.Objectrecognition4.Scenetextrecognition5.…..Learningtolearn,self-learning1.Adaptivelearning2.Context-drivenbabylearning3.Multi-modalityco-learning4.Human-likerobot5.…..DeepLearningtowardsTrue-AIBestframeworkfortrue-AIExcellentperformancesAdaptationtonewobservationsCross-taskknowledgesharingCross-modalityco-learningDNNCNNLSTMPartI:DeepLearningforAI[Brain-like]RecentHotAreasofDeepLearningNeuronNonlinearityPixel-to-PixelInferenceFeedbackStrategiesP2P:DeepPixel-to-PixelInferencePixel-wisepredictionInputOutput1.FullyConvolutionalNeuralNetworksmakesitpossible2.End-to-Endtrainable3.CanbeP2PdirectlyorP2P-awareDown-samplingUp-samplingP2P-awareFaceAlignment:Framework……ShapeSpaceDeconvKDown-samplingUp-samplingConvMaxPooling……DeConvConfidenceMapsforkeypoints…………FullyConnected0SFullyConnected1KS||||min00*SSS11KKKSSS||||min11*KKSSS001SSS0SDeconvKShape-IndexPooling0SFullyConnected1S||||min11*SSS112SSSShape-IndexPooling1S……Shape-IndexPooling1KSMethods300-WDatasetCommonChallengingFullsetZhuet.al[2012]8.2218.331.20RCPR[Burgos,2013]6.1817.268.35SDM[Xiong,2013]5.5715.407.50LBF[Ren,2014]4.9511.986.32LBFfast[Ren,2014]5.3815.507.37CFSS[Zhu,2015]4.739.985.76cGPRT[Lee,2015]--5.71Linkface4.808.605.54P2P-awareFaceAlignment4.198.425.02P2P-awareFaceAlignment:ResultsCommonsubsetP2P-awareFaceAlignment:ResultsChallengingsubsetP2P-awareFaceAlignment:ResultsNonlinearity:“NetworkinNetwork”(NIN)NIN:morebrain-like:complex-cellfilters,fullyconvolutionalIntuitivelylessoverfittingglobally,andmorediscriminativelocallyWithlessparameter#[4]IanJ.Goodfellow,DavidWarde-Farley,MehdiMirza,AaronC.Courville,YoshuaBengio:MaxoutNetworks.ICML(3)2013:1319-1327[4]CNNNINCanbeanysmallnetworks,e.g.MLP,Inceptionmodule,orothersforotherparticulartargets,butSMALL#featuremaps=#classesNonlinearity:GoogLeNet256480480512512512GoogLeNet=DeeperNetwork-in-NetworkInceptionmodules:small-networksofdifferentfiltersizesDeeperNINsonCASIA494kimagesof10ksubjects,followedbybinaryclassifierCurrentaccuracyonLFWis99.7%(saidtothereasonableupper-bound)LFWOrganizationAccuracyBaidu99.6%-99.8%Face++99.5%CUHK99.5%Facebook98.4%Nonlinearity:DeeperNINsforFaceRecognitionNonlinearity:DeeperNINsforFaceRecognition?SamePerson?Oursystemanswers“Yes”,Distance=8Threshold=200WeChatHersonsaidtheyareMummyandDaddy!Cross-borderofficersoftenchallengedher!FeedbackStrategiesFeedbackTypesCross-layerCross-taskRNN(slowinference)Contexts+FullyconvolutionalneuralnetworkCross-layercontext::multi-levelfeaturefusionGlobalimage-levelcontext::coherencebetweenpixel-wiselabellingandimagelabelpredictionLocalSuper-pixelcontext::within-superpixelconsistencyandcross-superpixelappearanceconsistencySemantic-edgecontext::Integratethecross-itemedgeforbetterdifferentiationFeedback:ContextualizedCNNforHumanParsingSun-glassUpper-clothesskirtscarfright-shoeright-legright-armpantsleft-shoeleft-legleft-armhatfacedressbeltbaghairnullMulti-resolutionfusionFeedback:ContextualizedCNN[Framework]TestPaperdollATRCo-CNNCo-CNNATRPaperdollTestFeedback:ContextualizedCNN[results]AccuracyForegroundaccuracyAverageprecisionAveragerecallAverageF-1scoresPaperdoll88.9662.1852.7549.4344.76Co-CNN95.2380.9081.5574.4276.95Co-CNN(+Chictopia10k)96.0283.5784.9577.6680.14Co-CNN(+Chictopia10k)(semantic-Edge)97.1888.8487.1284.0585.36w/oedgewedgeFeedback:ContextualizedCNN[results]OnlineHumanParsingEngine(0.15sor20fpssimplified)44.76%--85.36%:nearlyreadyformanyindustryapplications……PartII:DeepLearningforTrueAI[Baby-like]DeepLearningforTrueAISelf-learningSelf-improvedlearningduringinteractingwithrealworldThecoreiscontext/invarianceandwhereitisfromChappieDeepLearningforTrueAISingle-modalityAdaptationfrompreviousknowledgeContext-drivenbabylearningTemporalcontextsAssociationcontextsMulti-modalityMulti-modalitybabylearningVisualAudioNLPVGGModelVGGAdapt-BL:Scale-awareFastR-CNNfromVGG643312833256351233512L_convL_fcL_cls_score333input4096L_fc4096L_bbox_pred28cls_score28HeightWeight1RoIpoolingbbox_pred3512S_convS_fcS_cls_score4096S_fc4096S_bbox_pred2RoIpoolingLarge-scaleSub-networkSmall-scaleSub-network8Weight2Function..........Adapt-BL:Scale-awareFastR-CNNfromVGG16CaltechPedestrianDetectionBenchmarkPriorKnowledgeExploringPhysicalWorldwithContexts(context-basedself-learning)ExploringBroaderPhysicalWorld……FewpositiveinstancesContext-BL:ComputationalBabyLearning[Motivations]Mummy,isthisahorse?…………ExplorationsimulatedwithmassiveonlinevideosKnowledgeUpdateRicherPriorKnowledgeDetectorUpdateFurtherMiningVariableInstancesExemplar