Convolutional Network Features for Scene Recognition
Markus Koskela and Jorma Laaksonen. Convolutional Network
Features for Scene Recognition.
Accepted to the 22nd International Conference on Multimedia
(ACM MM 2014).
Orlando, Florida. November 2014.
DOI: http://dx.doi.org/10.1145/2647868.2655024
Scenes-15 database
Classification accuracies (and standard deviations) for the
scenes-15 dataset
| full image | spatial pyramid |
|---|
| 2010-aratio | 0.887 (0.003) | 0.915 (0.003) |
| 2010-distort | 0.881 (0.006) | 0.908 (0.003) |
| 2012-aratio | 0.887 (0.005) | 0.913 (0.005) |
| 2012-distort | 0.884 (0.006) | 0.907 (0.005) |
| fusion | 0.907 (0.003) | 0.921 (0.004) |
Confusion matrix for the spatial pyramid feature fusion experiment on the
scenes-15 dataset. Click the image for a larger version.
UIUC-sports database
Classification accuracies (and standard deviations) for the
uiuc-sports dataset
| full image | spatial pyramid |
|---|
| 2010-aratio | 0.935 (0.009) | 0.941 (0.007) |
| 2010-distort | 0.942 (0.010) | 0.944 (0.008) |
| 2012-aratio | 0.938 (0.008) | 0.946 (0.010) |
| 2012-distort | 0.938 (0.011) | 0.941 (0.006) |
| fusion | 0.947 (0.010) | 0.948 (0.009) |
Confusion matrix for the spatial pyramid feature fusion
experiment on the uiuc-sports dataset. Click the image for a larger version.
Indoor-67 database
Averages of per-class classification accuracies for the
indoor-67 dataset
| full image | spatial pyramid |
|---|
| 2010-aratio | 0.624 | 0.674 |
| 2010-distort | 0.601 | 0.657 |
| 2012-aratio | 0.627 | 0.669 |
| 2012-distort | 0.617 | 0.657 |
| fusion | 0.689 | 0.701 |
Categories of indoor-67 sorted by classification accuracy
with spatial pyramid feature fusion
| category | accuracy | category | accuracy | category | accuracy | category | accuracy |
|---|
| florist | 1.0 | garage | 0.833 | fastfood_restaurant | 0.706 | library | 0.600 |
| cloister | 1.0 | meeting_room | 0.818 | AVERAGE | 0.701 | bookstore | 0.600 |
| bowling | 1.0 | subway | 0.810 | prisoncell | 0.700 | locker_room | 0.571 |
| greenhouse | 0.950 | kitchen | 0.810 | gameroom | 0.700 | clothingstore | 0.556 |
| buffet | 0.950 | kindergarden | 0.800 | restaurant_kitchen | 0.696 | bar | 0.556 |
| casino | 0.947 | church_inside | 0.789 | operating_room | 0.684 | waitingroom | 0.524 |
| winecellar | 0.905 | gym | 0.778 | bakery | 0.684 | livingroom | 0.500 |
| elevator | 0.905 | classroom | 0.778 | dining_room | 0.667 | auditorium | 0.500 |
| concert_hall | 0.900 | bathroom | 0.778 | restaurant | 0.650 | artstudio | 0.500 |
| studiomusic | 0.895 | grocerystore | 0.762 | movietheater | 0.650 | shoeshop | 0.474 |
| closet | 0.889 | dentaloffice | 0.762 | mall | 0.650 | jewelleryshop | 0.455 |
| inside_subway | 0.857 | hospitalroom | 0.750 | laboratorywet | 0.636 | airport_inside | 0.450 |
| trainstation | 0.850 | inside_bus | 0.739 | warehouse | 0.619 | toystore | 0.409 |
| stairscase | 0.850 | videostore | 0.727 | corridor | 0.619 | deli | 0.368 |
| poolinside | 0.850 | laundromat | 0.727 | bedroom | 0.619 | office | 0.333 |
| pantry | 0.850 | computerroom | 0.722 | tv_studio | 0.611 | children_room | 0.278 |
| nursery | 0.850 | hairsalon | 0.714 | lobby | 0.600 | museum | 0.261 |
SUN-397 database
Classification accuracies (and standard deviations) for the
sun397 dataset
| full image | spatial pyramid |
|---|
| 2010-aratio | 0.456 (0.002) | 0.503 (0.001) |
| 2010-distort | 0.456 (0.004) | 0.504 (0.002) |
| 2012-aratio | 0.460 (0.002) | 0.506 (0.002) |
| 2012-distort | 0.454 (0.003) | 0.506 (0.003) |
| fusion | 0.519 (0.003) | 0.547 (0.002) |
Categories of sun397 sorted by classification accuracy
with spatial pyramid feature fusion
| category | accuracy (stdev) | category | accuracy (stdev) | category | accuracy (stdev) | category | accuracy (stdev) |
|---|
| underwater_coral_reef | 0.958 (0.026) | throne_room | 0.674 (0.062) | lock_chamber | 0.542 (0.076) | baggage_claim | 0.414 (0.061) |
| sky | 0.942 (0.015) | swimming_pool_outdoor | 0.672 (0.076) | bathroom | 0.540 (0.104) | booth_indoor | 0.414 (0.053) |
| oilrig | 0.910 (0.038) | ice_skating_rink_indoor | 0.670 (0.077) | cathedral_indoor | 0.540 (0.064) | market_indoor | 0.414 (0.039) |
| wave | 0.908 (0.052) | labyrinth_outdoor | 0.670 (0.050) | poolroom_establishment | 0.538 (0.084) | elevator_interior | 0.412 (0.059) |
| ball_pit | 0.902 (0.053) | laundromat | 0.670 (0.039) | dentists_office | 0.536 (0.051) | shopping_mall_indoor | 0.412 (0.064) |
| rock_arch | 0.894 (0.033) | swimming_pool_indoor | 0.668 (0.063) | aqueduct | 0.534 (0.064) | courtyard | 0.410 (0.067) |
| sauna | 0.894 (0.041) | hot_spring | 0.666 (0.068) | elevator_door | 0.534 (0.053) | locker_room | 0.408 (0.054) |
| riding_arena | 0.890 (0.029) | street | 0.666 (0.065) | temple_south_asia | 0.534 (0.080) | balcony_interior | 0.406 (0.054) |
| bullring | 0.888 (0.036) | ice_shelf | 0.664 (0.051) | dock | 0.532 (0.050) | youth_hostel | 0.404 (0.103) |
| raft | 0.868 (0.025) | fire_escape | 0.662 (0.053) | drugstore | 0.532 (0.061) | atrium_public | 0.400 (0.069) |
| batters_box | 0.864 (0.037) | heliport | 0.662 (0.048) | videostore | 0.528 (0.040) | hill | 0.400 (0.046) |
| stadium_baseball | 0.862 (0.058) | catacomb | 0.660 (0.070) | courtroom | 0.526 (0.067) | corral | 0.398 (0.048) |
| wind_farm | 0.856 (0.056) | fishpond | 0.660 (0.056) | subway_station_platform | 0.524 (0.077) | stage_indoor | 0.398 (0.070) |
| bamboo_forest | 0.850 (0.049) | slum | 0.660 (0.057) | boxing_ring | 0.522 (0.066) | bridge | 0.386 (0.065) |
| florist_shop_indoor | 0.840 (0.034) | martial_arts_gym | 0.650 (0.051) | canal_urban | 0.522 (0.076) | pharmacy | 0.384 (0.050) |
| greenhouse_outdoor | 0.836 (0.044) | canyon | 0.644 (0.073) | kindergarden_classroom | 0.522 (0.089) | schoolhouse | 0.382 (0.062) |
| volleyball_court_indoor | 0.836 (0.040) | escalator_indoor | 0.644 (0.056) | television_studio | 0.522 (0.091) | church_indoor | 0.378 (0.085) |
| landing_deck | 0.834 (0.053) | watering_hole | 0.644 (0.062) | van_interior | 0.522 (0.055) | bazaar_outdoor | 0.376 (0.043) |
| stadium_football | 0.832 (0.055) | wrestling_ring_indoor | 0.644 (0.047) | cemetery | 0.516 (0.060) | basilica | 0.374 (0.041) |
| car_interior_backseat | 0.830 (0.049) | medina | 0.642 (0.057) | cheese_factory | 0.516 (0.057) | boat_deck | 0.374 (0.056) |
| cockpit | 0.826 (0.046) | music_store | 0.640 (0.054) | dining_car | 0.516 (0.068) | shoe_shop | 0.374 (0.063) |
| podium_outdoor | 0.826 (0.030) | ski_resort | 0.640 (0.057) | hotel_room | 0.516 (0.100) | ballroom | 0.370 (0.062) |
| car_interior_frontseat | 0.820 (0.055) | stable | 0.640 (0.074) | ski_lodge | 0.516 (0.064) | lobby | 0.368 (0.149) |
| oast_house | 0.820 (0.041) | volleyball_court_outdoor | 0.640 (0.092) | snowfield | 0.516 (0.086) | pond | 0.368 (0.100) |
| carrousel | 0.818 (0.038) | railroad_track | 0.638 (0.061) | patio | 0.514 (0.050) | burial_chamber | 0.366 (0.077) |
| electrical_substation | 0.814 (0.048) | waterfall_fan | 0.636 (0.091) | planetarium_outdoor | 0.514 (0.079) | ticket_booth | 0.366 (0.092) |
| phone_booth | 0.814 (0.058) | kennel_indoor | 0.634 (0.061) | driving_range_outdoor | 0.512 (0.037) | pasture | 0.364 (0.065) |
| desert_sand | 0.806 (0.051) | tennis_court_outdoor | 0.634 (0.084) | yard | 0.512 (0.057) | field_wild | 0.362 (0.080) |
| windmill | 0.806 (0.059) | harbor | 0.630 (0.062) | moat_water | 0.510 (0.069) | home_office | 0.362 (0.088) |
| discotheque | 0.802 (0.059) | playground | 0.630 (0.067) | parlor | 0.510 (0.058) | building_facade | 0.360 (0.084) |
| volcano | 0.802 (0.026) | cottage_garden | 0.624 (0.068) | supermarket | 0.510 (0.070) | inn_outdoor | 0.360 (0.114) |
| cloister_indoor | 0.802 (0.047) | staircase | 0.622 (0.076) | hangar_indoor | 0.508 (0.058) | toyshop | 0.358 (0.089) |
| aquarium | 0.796 (0.049) | landfill | 0.618 (0.078) | picnic_area | 0.508 (0.083) | chalet | 0.356 (0.069) |
| pagoda | 0.796 (0.042) | viaduct | 0.618 (0.051) | tower | 0.506 (0.045) | pub_indoor | 0.352 (0.055) |
| tent_outdoor | 0.796 (0.067) | wheat_field | 0.618 (0.060) | forest_needleleaf | 0.504 (0.080) | auditorium | 0.350 (0.056) |
| igloo | 0.794 (0.025) | control_tower_outdoor | 0.616 (0.054) | computer_room | 0.502 (0.086) | airport_terminal | 0.350 (0.104) |
| skatepark | 0.794 (0.030) | cubicle_office | 0.616 (0.072) | park | 0.502 (0.063) | archive | 0.348 (0.051) |
| limousine_interior | 0.792 (0.051) | oil_refinery_outdoor | 0.614 (0.092) | playroom | 0.502 (0.045) | thriftshop | 0.346 (0.023) |
| bow_window_outdoor | 0.788 (0.082) | arch | 0.612 (0.066) | rice_paddy | 0.502 (0.073) | wet_bar | 0.344 (0.050) |
| tennis_court_indoor | 0.786 (0.042) | shopfront | 0.612 (0.047) | vineyard | 0.502 (0.046) | apartment_building_outdoor | 0.338 (0.089)
|
| shower | 0.782 (0.045) | waterfall_block | 0.612 (0.092) | beach | 0.500 (0.087) | classroom | 0.338 (0.069) |
| track_outdoor | 0.782 (0.047) | amusement_arcade | 0.610 (0.087) | bakery_shop | 0.498 (0.091) | library_indoor | 0.338 (0.076) |
| rope_bridge | 0.778 (0.058) | badlands | 0.610 (0.069) | creek | 0.498 (0.087) | office | 0.338 (0.074) |
| train_railway | 0.776 (0.053) | boardwalk | 0.610 (0.080) | valley | 0.498 (0.047) | candy_store | 0.336 (0.073) |
| squash_court | 0.774 (0.041) | doorway_outdoor | 0.610 (0.054) | art_studio | 0.494 (0.135) | hospital | 0.336 (0.078) |
| veterinarians_office | 0.774 (0.057) | ice_floe | 0.610 (0.092) | poolroom_home | 0.494 (0.031) | clean_room | 0.334 (0.054) |
| ocean | 0.770 (0.058) | barn | 0.608 (0.084) | reception | 0.494 (0.114) | monastery_outdoor | 0.334 (0.133) |
| hayfield | 0.762 (0.058) | boathouse | 0.608 (0.046) | server_room | 0.494 (0.062) | diner_indoor | 0.328 (0.101) |
| cavern_indoor | 0.756 (0.034) | formal_garden | 0.608 (0.097) | chicken_coop_indoor | 0.492 (0.051) | hotel_outdoor | 0.328 (0.103) |
| nursery | 0.756 (0.050) | observatory_outdoor | 0.608 (0.073) | corn_field | 0.490 (0.081) | campus | 0.326 (0.151) |
| tree_house | 0.752 (0.061) | temple_east_asia | 0.606 (0.045) | athletic_field_outdoor | 0.488 (0.096) | art_school | 0.324 (0.086) |
| sandbox | 0.750 (0.057) | orchard | 0.602 (0.064) | jail_cell | 0.488 (0.064) | construction_site | 0.322 (0.063) |
| wine_cellar_barrel_storage | 0.750 (0.086) | outhouse_outdoor | 0.602 (0.081) | ruin | 0.486 (0.083) | hunting_lodge_outdoor | 0.322 (0.074) |
| tree_farm | 0.746 (0.056) | movie_theater_indoor | 0.600 (0.043) | banquet_hall | 0.484 (0.059) | hospital_room | 0.318 (0.073) |
| covered_bridge_exterior | 0.744 (0.057) | butte | 0.598 (0.080) | jewelry_shop | 0.484 (0.095) | palace | 0.318 (0.092) |
| greenhouse_indoor | 0.742 (0.056) | ski_slope | 0.596 (0.048) | conference_room | 0.482 (0.079) | clothing_store | 0.316 (0.064) |
| promenade_deck | 0.740 (0.048) | butchers_shop | 0.594 (0.093) | cabin_outdoor | 0.482 (0.079) | mountain | 0.316 (0.067) |
| crevasse | 0.738 (0.063) | engine_room | 0.594 (0.053) | plaza | 0.482 (0.073) | museum_indoor | 0.316 (0.074) |
| lido_deck_outdoor | 0.738 (0.045) | mountain_snowy | 0.594 (0.048) | veranda | 0.482 (0.066) | coffee_shop | 0.312 (0.152) |
| pantry | 0.738 (0.065) | bus_interior | 0.592 (0.057) | putting_green | 0.480 (0.066) | general_store_outdoor | 0.312 (0.067) |
| theater_indoor_seats | 0.738 (0.055) | airplane_cabin | 0.590 (0.033) | beauty_salon | 0.478 (0.093) | industrial_area | 0.310 (0.055) |
| parking_garage_indoor | 0.730 (0.051) | herb_garden | 0.588 (0.093) | synagogue_outdoor | 0.478 (0.058) | restaurant_patio | 0.310 (0.078) |
| pulpit | 0.730 (0.043) | pilothouse_indoor | 0.588 (0.051) | forest_path | 0.476 (0.043) | canal_natural | 0.308 (0.065) |
| subway_interior | 0.730 (0.061) | dinette_home | 0.586 (0.057) | residential_neighborhood | 0.470 (0.071) | childs_room | 0.308 (0.055) |
| arrival_gate_outdoor | 0.728 (0.071) | jacuzzi_indoor | 0.586 (0.067) | firing_range_indoor | 0.468 (0.117) | castle | 0.306 (0.067) |
| lighthouse | 0.728 (0.037) | amphitheater | 0.584 (0.082) | fountain | 0.468 (0.045) | field_cultivated | 0.306 (0.057) |
| skyscraper | 0.726 (0.074) | badminton_court_indoor | 0.584 (0.072) | warehouse_indoor | 0.466 (0.055) | church_outdoor | 0.304 (0.076) |
| iceberg | 0.724 (0.072) | dinette_vehicle | 0.584 (0.082) | mausoleum | 0.464 (0.084) | physics_laboratory | 0.304 (0.076) |
| closet | 0.718 (0.048) | bookstore | 0.582 (0.077) | market_outdoor | 0.462 (0.040) | vegetable_garden | 0.304 (0.085) |
| waterfall_plunge | 0.718 (0.085) | gas_station | 0.580 (0.083) | dorm_room | 0.460 (0.136) | cathedral_outdoor | 0.302 (0.078) |
| bowling_alley | 0.710 (0.055) | balcony_exterior | 0.580 (0.089) | restaurant_kitchen | 0.460 (0.066) | library_outdoor | 0.302 (0.090) |
| driveway | 0.710 (0.069) | excavation | 0.580 (0.090) | garage_indoor | 0.458 (0.036) | food_court | 0.300 (0.062) |
| highway | 0.710 (0.053) | elevator_shaft | 0.578 (0.079) | campsite | 0.456 (0.051) | mosque_indoor | 0.300 (0.057) |
| trench | 0.710 (0.063) | hot_tub_outdoor | 0.576 (0.062) | chicken_coop_outdoor | 0.456 (0.034) | lecture_room | 0.292 (0.071) |
| auto_factory | 0.708 (0.079) | gazebo_exterior | 0.574 (0.086) | cliff | 0.456 (0.070) | fastfood_restaurant | 0.284 (0.079) |
| kasbah | 0.708 (0.082) | brewery_indoor | 0.572 (0.058) | art_gallery | 0.452 (0.115) | mansion | 0.282 (0.048) |
| anechoic_chamber | 0.706 (0.060) | waiting_room | 0.572 (0.122) | marsh | 0.450 (0.061) | dining_room | 0.280 (0.098) |
| dam | 0.704 (0.044) | barndoor | 0.572 (0.045) | garbage_dump | 0.448 (0.064) | ice_cream_parlor | 0.278 (0.036) |
| pavilion | 0.704 (0.056) | theater_indoor_procenium | 0.572 (0.082) | kitchen | 0.448 (0.064) | abbey | 0.276 (0.088) |
| train_station_platform | 0.704 (0.085) | crosswalk | 0.568 (0.045) | motel | 0.448 (0.135) | synagogue_indoor | 0.274 (0.054) |
| corridor | 0.702 (0.074) | jail_indoor | 0.568 (0.073) | sushi_bar | 0.448 (0.059) | chemistry_lab | 0.266 (0.040) |
| utility_room | 0.700 (0.051) | hangar_outdoor | 0.566 (0.059) | basketball_court_outdoor | 0.446 (0.049) | house | 0.266 (0.048) |
| water_tower | 0.700 (0.046) | apse_indoor | 0.564 (0.031) | amusement_park | 0.444 (0.048) | general_store_indoor | 0.256 (0.075) |
| forest_road | 0.696 (0.056) | toll_plaza | 0.564 (0.074) | parking_lot | 0.444 (0.053) | office_building | 0.256 (0.074) |
| runway | 0.694 (0.056) | casino_indoor | 0.562 (0.047) | berth | 0.442 (0.052) | botanical_garden | 0.254 (0.066) |
| parking_garage_outdoor | 0.692 (0.053) | cafeteria | 0.560 (0.079) | nuclear_power_plant_outdoor | 0.440 (0.065) | gift_shop | 0.254 (0.051) |
| raceway | 0.692 (0.049) | shed | 0.560 (0.057) | bazaar_indoor | 0.438 (0.040) | bar | 0.244 (0.064) |
| swamp | 0.690 (0.073) | control_room | 0.560 (0.060) | golf_course | 0.438 (0.066) | factory_indoor | 0.240 (0.087) |
| desert_vegetation | 0.688 (0.048) | conference_center | 0.558 (0.059) | living_room | 0.438 (0.142) | assembly_line | 0.238 (0.044) |
| manufactured_home | 0.688 (0.069) | fire_station | 0.558 (0.073) | attic | 0.436 (0.067) | bedroom | 0.238 (0.088) |
| galley | 0.686 (0.076) | islet | 0.558 (0.055) | kitchenette | 0.436 (0.049) | delicatessen | 0.226 (0.051) |
| lift_bridge | 0.686 (0.065) | podium_indoor | 0.558 (0.076) | coast | 0.430 (0.088) | river | 0.224 (0.084) |
| wine_cellar_bottle_storage | 0.686 (0.056) | operating_room | 0.554 (0.060) | baseball_field | 0.428 (0.104) | restaurant | 0.222 (0.082) |
| bow_window_indoor | 0.684 (0.051) | rainforest | 0.550 (0.052) | kennel_outdoor | 0.428 (0.056) | game_room | 0.218 (0.097) |
| gymnasium_indoor | 0.684 (0.069) | sandbar | 0.550 (0.076) | courthouse | 0.426 (0.045) | power_plant_outdoor | 0.210 (0.048) |
| racecourse | 0.684 (0.060) | mosque_outdoor | 0.548 (0.038) | fairway | 0.424 (0.059) | biology_laboratory | 0.200 (0.051) |
| ice_skating_rink_outdoor | 0.680 (0.075) | music_studio | 0.548 (0.070) | basement | 0.422 (0.051) | bistro_indoor | 0.186 (0.080) |
| sea_cliff | 0.678 (0.026) | AVERAGE | 0.547 (0.002) | diner_outdoor | 0.420 (0.060) | recreation_room | 0.166 (0.063) |
| topiary_garden | 0.678 (0.043) | alley | 0.542 (0.078) | lake_natural | 0.418 (0.066) | | |
| village | 0.678 (0.054) | bayou | 0.542 (0.082) | forest_broadleaf | 0.418 (0.057) | | |