{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Intake für die Datenwissenschaften\n",
    "\n",
    "Intake erleichtert das Laden vieler verschiedener Formate und Typen. Um einen vollständigen Überblick zu erhalten, schaut euch das [Plugin Directory](https://intake.readthedocs.io/en/latest/plugin-directory.html) und das [Intake Project Dashboard](https://intake.github.io/status/) an. Intake überführt die Daten dann in übliche Speicherformate wie Pandas DataFrames, Numpy-Arrays oder Python-Listen. Anschließend sind sie leicht durchsuchbar und auch für verteilte Systeme zugänglich. Sollte euch ein Plugin fehlen, könnt ihr auch selbst welche estellen, wie in [Making Drivers](https://intake.readthedocs.io/en/latest/making-plugins.html) beschrieben."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Laden einer Datenquelle\n",
    "\n",
    "Im Folgenden werden wir zwei csv-Datensätze lesen und in einen Intake-Katalog überführen. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-22T12:15:37.778250Z",
     "iopub.status.busy": "2026-05-22T12:15:37.778083Z",
     "iopub.status.idle": "2026-05-22T12:15:38.997719Z",
     "shell.execute_reply": "2026-05-22T12:15:38.997288Z",
     "shell.execute_reply.started": "2026-05-22T12:15:37.778232Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sources:\n",
      "  csv:\n",
      "    args:\n",
      "      urlpath: states_*.csv\n",
      "    description: ''\n",
      "    driver: intake.source.csv.CSVSource\n",
      "    metadata: {}\n",
      "\n"
     ]
    }
   ],
   "source": [
    "import intake\n",
    "\n",
    "\n",
    "ds = intake.open_csv(\"states_*.csv\")\n",
    "\n",
    "print(ds)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mit der `open_*`-Funktion von Intake lassen sich verschiedenen Datenquellen einlesen. Je nach Datenformat oder Dienst lassen sich unterschiedliche Argmuente verwenden."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Konfigurieren des Suchpfades für Datenquellen\n",
    "\n",
    "Intake überprüft die Intake-Konfigurationsdatei nach `catalog_pat`und die Umgebungsvariable `\"INTAKE_PATH\"` auf eine durch Doppelpunkte getrennte Liste von Pfaden bzw. Semikolon in Windows, um nach Katalogdateien zu suchen. Beim Import `intake` werden alle Einträge aus allen Katalogen angezeigt, auf die als Teil eines globalen Katalogs von `intake.cat` referenziert werden."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Daten lesen\n",
    "\n",
    "Intake liest Daten in Container verschiedener Formate:\n",
    "\n",
    "* Tabellen in Pandas DataFrames\n",
    "* Mehrdimensionale Array in Numpy Arrays\n",
    "* Semistrukturierte Daten in Python-Listen von Objekten, üblicherweise Dictionaries\n",
    "\n",
    "Um herauszufinden, in welchem Containerformat Intake die Daten vorhält, könnt ihr das `container`-Attribut verwenden:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-22T12:15:38.998565Z",
     "iopub.status.busy": "2026-05-22T12:15:38.998335Z",
     "iopub.status.idle": "2026-05-22T12:15:39.002087Z",
     "shell.execute_reply": "2026-05-22T12:15:39.001775Z",
     "shell.execute_reply.started": "2026-05-22T12:15:38.998546Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'dataframe'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ds.container"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Neben `dataframe` kann das Ergebnis auch `ndarray` oder `python` sein."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-22T12:15:39.002563Z",
     "iopub.status.busy": "2026-05-22T12:15:39.002449Z",
     "iopub.status.idle": "2026-05-22T12:15:39.012044Z",
     "shell.execute_reply": "2026-05-22T12:15:39.011774Z",
     "shell.execute_reply.started": "2026-05-22T12:15:39.002553Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>state</th>\n",
       "      <th>slug</th>\n",
       "      <th>code</th>\n",
       "      <th>nickname</th>\n",
       "      <th>website</th>\n",
       "      <th>admission_date</th>\n",
       "      <th>admission_number</th>\n",
       "      <th>capital_city</th>\n",
       "      <th>capital_url</th>\n",
       "      <th>population</th>\n",
       "      <th>population_rank</th>\n",
       "      <th>constitution_url</th>\n",
       "      <th>state_flag_url</th>\n",
       "      <th>state_seal_url</th>\n",
       "      <th>map_image_url</th>\n",
       "      <th>landscape_background_url</th>\n",
       "      <th>skyline_background_url</th>\n",
       "      <th>twitter_url</th>\n",
       "      <th>facebook_url</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Alabama</td>\n",
       "      <td>alabama</td>\n",
       "      <td>AL</td>\n",
       "      <td>Yellowhammer State</td>\n",
       "      <td>http://www.alabama.gov</td>\n",
       "      <td>1819-12-14</td>\n",
       "      <td>22</td>\n",
       "      <td>Montgomery</td>\n",
       "      <td>http://www.montgomeryal.gov</td>\n",
       "      <td>4833722</td>\n",
       "      <td>23</td>\n",
       "      <td>http://alisondb.legislature.state.al.us/alison...</td>\n",
       "      <td>https://cdn.civil.services/us-states/flags/ala...</td>\n",
       "      <td>https://cdn.civil.services/us-states/seals/ala...</td>\n",
       "      <td>https://cdn.civil.services/us-states/maps/alab...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://twitter.com/alabamagov</td>\n",
       "      <td>https://www.facebook.com/alabamagov</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Alaska</td>\n",
       "      <td>alaska</td>\n",
       "      <td>AK</td>\n",
       "      <td>The Last Frontier</td>\n",
       "      <td>http://alaska.gov</td>\n",
       "      <td>1959-01-03</td>\n",
       "      <td>49</td>\n",
       "      <td>Juneau</td>\n",
       "      <td>http://www.juneau.org</td>\n",
       "      <td>735132</td>\n",
       "      <td>47</td>\n",
       "      <td>http://www.legis.state.ak.us/basis/folioproxy....</td>\n",
       "      <td>https://cdn.civil.services/us-states/flags/ala...</td>\n",
       "      <td>https://cdn.civil.services/us-states/seals/ala...</td>\n",
       "      <td>https://cdn.civil.services/us-states/maps/alas...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://twitter.com/alaska</td>\n",
       "      <td>https://www.facebook.com/AlaskaLocalGovernments</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Arizona</td>\n",
       "      <td>arizona</td>\n",
       "      <td>AZ</td>\n",
       "      <td>The Grand Canyon State</td>\n",
       "      <td>https://az.gov</td>\n",
       "      <td>1912-02-14</td>\n",
       "      <td>48</td>\n",
       "      <td>Phoenix</td>\n",
       "      <td>https://www.phoenix.gov</td>\n",
       "      <td>6626624</td>\n",
       "      <td>15</td>\n",
       "      <td>http://www.azleg.gov/Constitution.asp</td>\n",
       "      <td>https://cdn.civil.services/us-states/flags/ari...</td>\n",
       "      <td>https://cdn.civil.services/us-states/seals/ari...</td>\n",
       "      <td>https://cdn.civil.services/us-states/maps/ariz...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Arkansas</td>\n",
       "      <td>arkansas</td>\n",
       "      <td>AR</td>\n",
       "      <td>The Natural State</td>\n",
       "      <td>http://arkansas.gov</td>\n",
       "      <td>1836-06-15</td>\n",
       "      <td>25</td>\n",
       "      <td>Little Rock</td>\n",
       "      <td>http://www.littlerock.org</td>\n",
       "      <td>2959373</td>\n",
       "      <td>32</td>\n",
       "      <td>http://www.arkleg.state.ar.us/assembly/Summary...</td>\n",
       "      <td>https://cdn.civil.services/us-states/flags/ark...</td>\n",
       "      <td>https://cdn.civil.services/us-states/seals/ark...</td>\n",
       "      <td>https://cdn.civil.services/us-states/maps/arka...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://twitter.com/arkansasgov</td>\n",
       "      <td>https://www.facebook.com/Arkansas.gov</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>California</td>\n",
       "      <td>california</td>\n",
       "      <td>CA</td>\n",
       "      <td>Golden State</td>\n",
       "      <td>http://www.ca.gov</td>\n",
       "      <td>1850-09-09</td>\n",
       "      <td>31</td>\n",
       "      <td>Sacramento</td>\n",
       "      <td>http://www.cityofsacramento.org</td>\n",
       "      <td>38332521</td>\n",
       "      <td>1</td>\n",
       "      <td>http://www.leginfo.ca.gov/const-toc.html</td>\n",
       "      <td>https://cdn.civil.services/us-states/flags/cal...</td>\n",
       "      <td>https://cdn.civil.services/us-states/seals/cal...</td>\n",
       "      <td>https://cdn.civil.services/us-states/maps/cali...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://twitter.com/cagovernment</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        state        slug code                nickname  \\\n",
       "0     Alabama     alabama   AL      Yellowhammer State   \n",
       "1      Alaska      alaska   AK       The Last Frontier   \n",
       "2     Arizona     arizona   AZ  The Grand Canyon State   \n",
       "3    Arkansas    arkansas   AR       The Natural State   \n",
       "4  California  california   CA            Golden State   \n",
       "\n",
       "                  website admission_date  admission_number capital_city  \\\n",
       "0  http://www.alabama.gov     1819-12-14                22   Montgomery   \n",
       "1       http://alaska.gov     1959-01-03                49       Juneau   \n",
       "2          https://az.gov     1912-02-14                48      Phoenix   \n",
       "3     http://arkansas.gov     1836-06-15                25  Little Rock   \n",
       "4       http://www.ca.gov     1850-09-09                31   Sacramento   \n",
       "\n",
       "                       capital_url  population  population_rank  \\\n",
       "0      http://www.montgomeryal.gov     4833722               23   \n",
       "1            http://www.juneau.org      735132               47   \n",
       "2          https://www.phoenix.gov     6626624               15   \n",
       "3        http://www.littlerock.org     2959373               32   \n",
       "4  http://www.cityofsacramento.org    38332521                1   \n",
       "\n",
       "                                    constitution_url  \\\n",
       "0  http://alisondb.legislature.state.al.us/alison...   \n",
       "1  http://www.legis.state.ak.us/basis/folioproxy....   \n",
       "2              http://www.azleg.gov/Constitution.asp   \n",
       "3  http://www.arkleg.state.ar.us/assembly/Summary...   \n",
       "4           http://www.leginfo.ca.gov/const-toc.html   \n",
       "\n",
       "                                      state_flag_url  \\\n",
       "0  https://cdn.civil.services/us-states/flags/ala...   \n",
       "1  https://cdn.civil.services/us-states/flags/ala...   \n",
       "2  https://cdn.civil.services/us-states/flags/ari...   \n",
       "3  https://cdn.civil.services/us-states/flags/ark...   \n",
       "4  https://cdn.civil.services/us-states/flags/cal...   \n",
       "\n",
       "                                      state_seal_url  \\\n",
       "0  https://cdn.civil.services/us-states/seals/ala...   \n",
       "1  https://cdn.civil.services/us-states/seals/ala...   \n",
       "2  https://cdn.civil.services/us-states/seals/ari...   \n",
       "3  https://cdn.civil.services/us-states/seals/ark...   \n",
       "4  https://cdn.civil.services/us-states/seals/cal...   \n",
       "\n",
       "                                       map_image_url  \\\n",
       "0  https://cdn.civil.services/us-states/maps/alab...   \n",
       "1  https://cdn.civil.services/us-states/maps/alas...   \n",
       "2  https://cdn.civil.services/us-states/maps/ariz...   \n",
       "3  https://cdn.civil.services/us-states/maps/arka...   \n",
       "4  https://cdn.civil.services/us-states/maps/cali...   \n",
       "\n",
       "                            landscape_background_url  \\\n",
       "0  https://cdn.civil.services/us-states/backgroun...   \n",
       "1  https://cdn.civil.services/us-states/backgroun...   \n",
       "2  https://cdn.civil.services/us-states/backgroun...   \n",
       "3  https://cdn.civil.services/us-states/backgroun...   \n",
       "4  https://cdn.civil.services/us-states/backgroun...   \n",
       "\n",
       "                              skyline_background_url  \\\n",
       "0  https://cdn.civil.services/us-states/backgroun...   \n",
       "1  https://cdn.civil.services/us-states/backgroun...   \n",
       "2  https://cdn.civil.services/us-states/backgroun...   \n",
       "3  https://cdn.civil.services/us-states/backgroun...   \n",
       "4  https://cdn.civil.services/us-states/backgroun...   \n",
       "\n",
       "                        twitter_url  \\\n",
       "0    https://twitter.com/alabamagov   \n",
       "1        https://twitter.com/alaska   \n",
       "2                               NaN   \n",
       "3   https://twitter.com/arkansasgov   \n",
       "4  https://twitter.com/cagovernment   \n",
       "\n",
       "                                      facebook_url  \n",
       "0              https://www.facebook.com/alabamagov  \n",
       "1  https://www.facebook.com/AlaskaLocalGovernments  \n",
       "2                                              NaN  \n",
       "3            https://www.facebook.com/Arkansas.gov  \n",
       "4                                              NaN  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = ds.read()\n",
    "\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-22T12:15:39.012561Z",
     "iopub.status.busy": "2026-05-22T12:15:39.012439Z",
     "iopub.status.idle": "2026-05-22T12:15:39.014184Z",
     "shell.execute_reply": "2026-05-22T12:15:39.013815Z",
     "shell.execute_reply.started": "2026-05-22T12:15:39.012551Z"
    }
   },
   "outputs": [],
   "source": [
    "for chunk in ds.read_chunked():\n",
    "    print(f\"Chunk: {len(chunk)}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-22T12:15:39.014818Z",
     "iopub.status.busy": "2026-05-22T12:15:39.014624Z",
     "iopub.status.idle": "2026-05-22T12:15:39.431652Z",
     "shell.execute_reply": "2026-05-22T12:15:39.431243Z",
     "shell.execute_reply.started": "2026-05-22T12:15:39.014730Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>state</th>\n",
       "      <th>slug</th>\n",
       "      <th>code</th>\n",
       "      <th>nickname</th>\n",
       "      <th>website</th>\n",
       "      <th>admission_date</th>\n",
       "      <th>admission_number</th>\n",
       "      <th>capital_city</th>\n",
       "      <th>capital_url</th>\n",
       "      <th>population</th>\n",
       "      <th>population_rank</th>\n",
       "      <th>constitution_url</th>\n",
       "      <th>state_flag_url</th>\n",
       "      <th>state_seal_url</th>\n",
       "      <th>map_image_url</th>\n",
       "      <th>landscape_background_url</th>\n",
       "      <th>skyline_background_url</th>\n",
       "      <th>twitter_url</th>\n",
       "      <th>facebook_url</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Alabama</td>\n",
       "      <td>alabama</td>\n",
       "      <td>AL</td>\n",
       "      <td>Yellowhammer State</td>\n",
       "      <td>http://www.alabama.gov</td>\n",
       "      <td>1819-12-14</td>\n",
       "      <td>22</td>\n",
       "      <td>Montgomery</td>\n",
       "      <td>http://www.montgomeryal.gov</td>\n",
       "      <td>4833722</td>\n",
       "      <td>23</td>\n",
       "      <td>http://alisondb.legislature.state.al.us/alison...</td>\n",
       "      <td>https://cdn.civil.services/us-states/flags/ala...</td>\n",
       "      <td>https://cdn.civil.services/us-states/seals/ala...</td>\n",
       "      <td>https://cdn.civil.services/us-states/maps/alab...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://twitter.com/alabamagov</td>\n",
       "      <td>https://www.facebook.com/alabamagov</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Alaska</td>\n",
       "      <td>alaska</td>\n",
       "      <td>AK</td>\n",
       "      <td>The Last Frontier</td>\n",
       "      <td>http://alaska.gov</td>\n",
       "      <td>1959-01-03</td>\n",
       "      <td>49</td>\n",
       "      <td>Juneau</td>\n",
       "      <td>http://www.juneau.org</td>\n",
       "      <td>735132</td>\n",
       "      <td>47</td>\n",
       "      <td>http://www.legis.state.ak.us/basis/folioproxy....</td>\n",
       "      <td>https://cdn.civil.services/us-states/flags/ala...</td>\n",
       "      <td>https://cdn.civil.services/us-states/seals/ala...</td>\n",
       "      <td>https://cdn.civil.services/us-states/maps/alas...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://twitter.com/alaska</td>\n",
       "      <td>https://www.facebook.com/AlaskaLocalGovernments</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Arizona</td>\n",
       "      <td>arizona</td>\n",
       "      <td>AZ</td>\n",
       "      <td>The Grand Canyon State</td>\n",
       "      <td>https://az.gov</td>\n",
       "      <td>1912-02-14</td>\n",
       "      <td>48</td>\n",
       "      <td>Phoenix</td>\n",
       "      <td>https://www.phoenix.gov</td>\n",
       "      <td>6626624</td>\n",
       "      <td>15</td>\n",
       "      <td>http://www.azleg.gov/Constitution.asp</td>\n",
       "      <td>https://cdn.civil.services/us-states/flags/ari...</td>\n",
       "      <td>https://cdn.civil.services/us-states/seals/ari...</td>\n",
       "      <td>https://cdn.civil.services/us-states/maps/ariz...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Arkansas</td>\n",
       "      <td>arkansas</td>\n",
       "      <td>AR</td>\n",
       "      <td>The Natural State</td>\n",
       "      <td>http://arkansas.gov</td>\n",
       "      <td>1836-06-15</td>\n",
       "      <td>25</td>\n",
       "      <td>Little Rock</td>\n",
       "      <td>http://www.littlerock.org</td>\n",
       "      <td>2959373</td>\n",
       "      <td>32</td>\n",
       "      <td>http://www.arkleg.state.ar.us/assembly/Summary...</td>\n",
       "      <td>https://cdn.civil.services/us-states/flags/ark...</td>\n",
       "      <td>https://cdn.civil.services/us-states/seals/ark...</td>\n",
       "      <td>https://cdn.civil.services/us-states/maps/arka...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://twitter.com/arkansasgov</td>\n",
       "      <td>https://www.facebook.com/Arkansas.gov</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>California</td>\n",
       "      <td>california</td>\n",
       "      <td>CA</td>\n",
       "      <td>Golden State</td>\n",
       "      <td>http://www.ca.gov</td>\n",
       "      <td>1850-09-09</td>\n",
       "      <td>31</td>\n",
       "      <td>Sacramento</td>\n",
       "      <td>http://www.cityofsacramento.org</td>\n",
       "      <td>38332521</td>\n",
       "      <td>1</td>\n",
       "      <td>http://www.leginfo.ca.gov/const-toc.html</td>\n",
       "      <td>https://cdn.civil.services/us-states/flags/cal...</td>\n",
       "      <td>https://cdn.civil.services/us-states/seals/cal...</td>\n",
       "      <td>https://cdn.civil.services/us-states/maps/cali...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://twitter.com/cagovernment</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        state        slug code                nickname  \\\n",
       "0     Alabama     alabama   AL      Yellowhammer State   \n",
       "1      Alaska      alaska   AK       The Last Frontier   \n",
       "2     Arizona     arizona   AZ  The Grand Canyon State   \n",
       "3    Arkansas    arkansas   AR       The Natural State   \n",
       "4  California  california   CA            Golden State   \n",
       "\n",
       "                  website admission_date  admission_number capital_city  \\\n",
       "0  http://www.alabama.gov     1819-12-14                22   Montgomery   \n",
       "1       http://alaska.gov     1959-01-03                49       Juneau   \n",
       "2          https://az.gov     1912-02-14                48      Phoenix   \n",
       "3     http://arkansas.gov     1836-06-15                25  Little Rock   \n",
       "4       http://www.ca.gov     1850-09-09                31   Sacramento   \n",
       "\n",
       "                       capital_url  population  population_rank  \\\n",
       "0      http://www.montgomeryal.gov     4833722               23   \n",
       "1            http://www.juneau.org      735132               47   \n",
       "2          https://www.phoenix.gov     6626624               15   \n",
       "3        http://www.littlerock.org     2959373               32   \n",
       "4  http://www.cityofsacramento.org    38332521                1   \n",
       "\n",
       "                                    constitution_url  \\\n",
       "0  http://alisondb.legislature.state.al.us/alison...   \n",
       "1  http://www.legis.state.ak.us/basis/folioproxy....   \n",
       "2              http://www.azleg.gov/Constitution.asp   \n",
       "3  http://www.arkleg.state.ar.us/assembly/Summary...   \n",
       "4           http://www.leginfo.ca.gov/const-toc.html   \n",
       "\n",
       "                                      state_flag_url  \\\n",
       "0  https://cdn.civil.services/us-states/flags/ala...   \n",
       "1  https://cdn.civil.services/us-states/flags/ala...   \n",
       "2  https://cdn.civil.services/us-states/flags/ari...   \n",
       "3  https://cdn.civil.services/us-states/flags/ark...   \n",
       "4  https://cdn.civil.services/us-states/flags/cal...   \n",
       "\n",
       "                                      state_seal_url  \\\n",
       "0  https://cdn.civil.services/us-states/seals/ala...   \n",
       "1  https://cdn.civil.services/us-states/seals/ala...   \n",
       "2  https://cdn.civil.services/us-states/seals/ari...   \n",
       "3  https://cdn.civil.services/us-states/seals/ark...   \n",
       "4  https://cdn.civil.services/us-states/seals/cal...   \n",
       "\n",
       "                                       map_image_url  \\\n",
       "0  https://cdn.civil.services/us-states/maps/alab...   \n",
       "1  https://cdn.civil.services/us-states/maps/alas...   \n",
       "2  https://cdn.civil.services/us-states/maps/ariz...   \n",
       "3  https://cdn.civil.services/us-states/maps/arka...   \n",
       "4  https://cdn.civil.services/us-states/maps/cali...   \n",
       "\n",
       "                            landscape_background_url  \\\n",
       "0  https://cdn.civil.services/us-states/backgroun...   \n",
       "1  https://cdn.civil.services/us-states/backgroun...   \n",
       "2  https://cdn.civil.services/us-states/backgroun...   \n",
       "3  https://cdn.civil.services/us-states/backgroun...   \n",
       "4  https://cdn.civil.services/us-states/backgroun...   \n",
       "\n",
       "                              skyline_background_url  \\\n",
       "0  https://cdn.civil.services/us-states/backgroun...   \n",
       "1  https://cdn.civil.services/us-states/backgroun...   \n",
       "2  https://cdn.civil.services/us-states/backgroun...   \n",
       "3  https://cdn.civil.services/us-states/backgroun...   \n",
       "4  https://cdn.civil.services/us-states/backgroun...   \n",
       "\n",
       "                        twitter_url  \\\n",
       "0    https://twitter.com/alabamagov   \n",
       "1        https://twitter.com/alaska   \n",
       "2                               NaN   \n",
       "3   https://twitter.com/arkansasgov   \n",
       "4  https://twitter.com/cagovernment   \n",
       "\n",
       "                                      facebook_url  \n",
       "0              https://www.facebook.com/alabamagov  \n",
       "1  https://www.facebook.com/AlaskaLocalGovernments  \n",
       "2                                              NaN  \n",
       "3            https://www.facebook.com/Arkansas.gov  \n",
       "4                                              NaN  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ddf = ds.to_dask()\n",
    "\n",
    "ddf.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-22T12:15:39.433296Z",
     "iopub.status.busy": "2026-05-22T12:15:39.433159Z",
     "iopub.status.idle": "2026-05-22T12:15:39.435795Z",
     "shell.execute_reply": "2026-05-22T12:15:39.435500Z",
     "shell.execute_reply.started": "2026-05-22T12:15:39.433286Z"
    }
   },
   "outputs": [],
   "source": [
    "cat = intake.open_catalog(\"us_states.yml\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-22T12:15:39.436147Z",
     "iopub.status.busy": "2026-05-22T12:15:39.436073Z",
     "iopub.status.idle": "2026-05-22T12:15:39.438418Z",
     "shell.execute_reply": "2026-05-22T12:15:39.438139Z",
     "shell.execute_reply.started": "2026-05-22T12:15:39.436140Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['states']"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(cat)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-22T12:15:39.438923Z",
     "iopub.status.busy": "2026-05-22T12:15:39.438844Z",
     "iopub.status.idle": "2026-05-22T12:15:39.516935Z",
     "shell.execute_reply": "2026-05-22T12:15:39.516633Z",
     "shell.execute_reply.started": "2026-05-22T12:15:39.438916Z"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>state</th>\n",
       "      <th>slug</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Alabama</td>\n",
       "      <td>alabama</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Alaska</td>\n",
       "      <td>alaska</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Arizona</td>\n",
       "      <td>arizona</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Arkansas</td>\n",
       "      <td>arkansas</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>California</td>\n",
       "      <td>california</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        state        slug\n",
       "0     Alabama     alabama\n",
       "1      Alaska      alaska\n",
       "2     Arizona     arizona\n",
       "3    Arkansas    arkansas\n",
       "4  California  california"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cat.states.to_dask()[[\"state\", \"slug\"]].head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2026-05-22T12:15:39.517537Z",
     "iopub.status.busy": "2026-05-22T12:15:39.517441Z",
     "iopub.status.idle": "2026-05-22T12:15:39.526754Z",
     "shell.execute_reply": "2026-05-22T12:15:39.526499Z",
     "shell.execute_reply.started": "2026-05-22T12:15:39.517528Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "      <th>1</th>\n",
       "      <th>2</th>\n",
       "      <th>3</th>\n",
       "      <th>4</th>\n",
       "      <th>5</th>\n",
       "      <th>6</th>\n",
       "      <th>7</th>\n",
       "      <th>8</th>\n",
       "      <th>9</th>\n",
       "      <th>10</th>\n",
       "      <th>11</th>\n",
       "      <th>12</th>\n",
       "      <th>13</th>\n",
       "      <th>14</th>\n",
       "      <th>15</th>\n",
       "      <th>16</th>\n",
       "      <th>17</th>\n",
       "      <th>18</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Alabama</td>\n",
       "      <td>alabama</td>\n",
       "      <td>AL</td>\n",
       "      <td>Yellowhammer State</td>\n",
       "      <td>http://www.alabama.gov</td>\n",
       "      <td>1819-12-14</td>\n",
       "      <td>22</td>\n",
       "      <td>Montgomery</td>\n",
       "      <td>http://www.montgomeryal.gov</td>\n",
       "      <td>4833722</td>\n",
       "      <td>23</td>\n",
       "      <td>http://alisondb.legislature.state.al.us/alison...</td>\n",
       "      <td>https://cdn.civil.services/us-states/flags/ala...</td>\n",
       "      <td>https://cdn.civil.services/us-states/seals/ala...</td>\n",
       "      <td>https://cdn.civil.services/us-states/maps/alab...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://twitter.com/alabamagov</td>\n",
       "      <td>https://www.facebook.com/alabamagov</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Alaska</td>\n",
       "      <td>alaska</td>\n",
       "      <td>AK</td>\n",
       "      <td>The Last Frontier</td>\n",
       "      <td>http://alaska.gov</td>\n",
       "      <td>1959-01-03</td>\n",
       "      <td>49</td>\n",
       "      <td>Juneau</td>\n",
       "      <td>http://www.juneau.org</td>\n",
       "      <td>735132</td>\n",
       "      <td>47</td>\n",
       "      <td>http://www.legis.state.ak.us/basis/folioproxy....</td>\n",
       "      <td>https://cdn.civil.services/us-states/flags/ala...</td>\n",
       "      <td>https://cdn.civil.services/us-states/seals/ala...</td>\n",
       "      <td>https://cdn.civil.services/us-states/maps/alas...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://twitter.com/alaska</td>\n",
       "      <td>https://www.facebook.com/AlaskaLocalGovernments</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Arizona</td>\n",
       "      <td>arizona</td>\n",
       "      <td>AZ</td>\n",
       "      <td>The Grand Canyon State</td>\n",
       "      <td>https://az.gov</td>\n",
       "      <td>1912-02-14</td>\n",
       "      <td>48</td>\n",
       "      <td>Phoenix</td>\n",
       "      <td>https://www.phoenix.gov</td>\n",
       "      <td>6626624</td>\n",
       "      <td>15</td>\n",
       "      <td>http://www.azleg.gov/Constitution.asp</td>\n",
       "      <td>https://cdn.civil.services/us-states/flags/ari...</td>\n",
       "      <td>https://cdn.civil.services/us-states/seals/ari...</td>\n",
       "      <td>https://cdn.civil.services/us-states/maps/ariz...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Arkansas</td>\n",
       "      <td>arkansas</td>\n",
       "      <td>AR</td>\n",
       "      <td>The Natural State</td>\n",
       "      <td>http://arkansas.gov</td>\n",
       "      <td>1836-06-15</td>\n",
       "      <td>25</td>\n",
       "      <td>Little Rock</td>\n",
       "      <td>http://www.littlerock.org</td>\n",
       "      <td>2959373</td>\n",
       "      <td>32</td>\n",
       "      <td>http://www.arkleg.state.ar.us/assembly/Summary...</td>\n",
       "      <td>https://cdn.civil.services/us-states/flags/ark...</td>\n",
       "      <td>https://cdn.civil.services/us-states/seals/ark...</td>\n",
       "      <td>https://cdn.civil.services/us-states/maps/arka...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://twitter.com/arkansasgov</td>\n",
       "      <td>https://www.facebook.com/Arkansas.gov</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>California</td>\n",
       "      <td>california</td>\n",
       "      <td>CA</td>\n",
       "      <td>Golden State</td>\n",
       "      <td>http://www.ca.gov</td>\n",
       "      <td>1850-09-09</td>\n",
       "      <td>31</td>\n",
       "      <td>Sacramento</td>\n",
       "      <td>http://www.cityofsacramento.org</td>\n",
       "      <td>38332521</td>\n",
       "      <td>1</td>\n",
       "      <td>http://www.leginfo.ca.gov/const-toc.html</td>\n",
       "      <td>https://cdn.civil.services/us-states/flags/cal...</td>\n",
       "      <td>https://cdn.civil.services/us-states/seals/cal...</td>\n",
       "      <td>https://cdn.civil.services/us-states/maps/cali...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://cdn.civil.services/us-states/backgroun...</td>\n",
       "      <td>https://twitter.com/cagovernment</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           0           1   2                       3                       4   \\\n",
       "0     Alabama     alabama  AL      Yellowhammer State  http://www.alabama.gov   \n",
       "1      Alaska      alaska  AK       The Last Frontier       http://alaska.gov   \n",
       "2     Arizona     arizona  AZ  The Grand Canyon State          https://az.gov   \n",
       "3    Arkansas    arkansas  AR       The Natural State     http://arkansas.gov   \n",
       "4  California  california  CA            Golden State       http://www.ca.gov   \n",
       "\n",
       "           5   6            7                                8         9   10  \\\n",
       "0  1819-12-14  22   Montgomery      http://www.montgomeryal.gov   4833722  23   \n",
       "1  1959-01-03  49       Juneau            http://www.juneau.org    735132  47   \n",
       "2  1912-02-14  48      Phoenix          https://www.phoenix.gov   6626624  15   \n",
       "3  1836-06-15  25  Little Rock        http://www.littlerock.org   2959373  32   \n",
       "4  1850-09-09  31   Sacramento  http://www.cityofsacramento.org  38332521   1   \n",
       "\n",
       "                                                  11  \\\n",
       "0  http://alisondb.legislature.state.al.us/alison...   \n",
       "1  http://www.legis.state.ak.us/basis/folioproxy....   \n",
       "2              http://www.azleg.gov/Constitution.asp   \n",
       "3  http://www.arkleg.state.ar.us/assembly/Summary...   \n",
       "4           http://www.leginfo.ca.gov/const-toc.html   \n",
       "\n",
       "                                                  12  \\\n",
       "0  https://cdn.civil.services/us-states/flags/ala...   \n",
       "1  https://cdn.civil.services/us-states/flags/ala...   \n",
       "2  https://cdn.civil.services/us-states/flags/ari...   \n",
       "3  https://cdn.civil.services/us-states/flags/ark...   \n",
       "4  https://cdn.civil.services/us-states/flags/cal...   \n",
       "\n",
       "                                                  13  \\\n",
       "0  https://cdn.civil.services/us-states/seals/ala...   \n",
       "1  https://cdn.civil.services/us-states/seals/ala...   \n",
       "2  https://cdn.civil.services/us-states/seals/ari...   \n",
       "3  https://cdn.civil.services/us-states/seals/ark...   \n",
       "4  https://cdn.civil.services/us-states/seals/cal...   \n",
       "\n",
       "                                                  14  \\\n",
       "0  https://cdn.civil.services/us-states/maps/alab...   \n",
       "1  https://cdn.civil.services/us-states/maps/alas...   \n",
       "2  https://cdn.civil.services/us-states/maps/ariz...   \n",
       "3  https://cdn.civil.services/us-states/maps/arka...   \n",
       "4  https://cdn.civil.services/us-states/maps/cali...   \n",
       "\n",
       "                                                  15  \\\n",
       "0  https://cdn.civil.services/us-states/backgroun...   \n",
       "1  https://cdn.civil.services/us-states/backgroun...   \n",
       "2  https://cdn.civil.services/us-states/backgroun...   \n",
       "3  https://cdn.civil.services/us-states/backgroun...   \n",
       "4  https://cdn.civil.services/us-states/backgroun...   \n",
       "\n",
       "                                                  16  \\\n",
       "0  https://cdn.civil.services/us-states/backgroun...   \n",
       "1  https://cdn.civil.services/us-states/backgroun...   \n",
       "2  https://cdn.civil.services/us-states/backgroun...   \n",
       "3  https://cdn.civil.services/us-states/backgroun...   \n",
       "4  https://cdn.civil.services/us-states/backgroun...   \n",
       "\n",
       "                                 17  \\\n",
       "0    https://twitter.com/alabamagov   \n",
       "1        https://twitter.com/alaska   \n",
       "2                               NaN   \n",
       "3   https://twitter.com/arkansasgov   \n",
       "4  https://twitter.com/cagovernment   \n",
       "\n",
       "                                                18  \n",
       "0              https://www.facebook.com/alabamagov  \n",
       "1  https://www.facebook.com/AlaskaLocalGovernments  \n",
       "2                                              NaN  \n",
       "3            https://www.facebook.com/Arkansas.gov  \n",
       "4                                              NaN  "
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cat.states(csv_kwargs={\"header\": None, \"skiprows\": 1}).read().head()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.13 Kernel",
   "language": "python",
   "name": "python313"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.0"
  },
  "latex_envs": {
   "LaTeX_envs_menu_present": true,
   "autoclose": false,
   "autocomplete": true,
   "bibliofile": "biblio.bib",
   "cite_by": "apalike",
   "current_citInitial": 1,
   "eqLabelWithNumbers": true,
   "eqNumInitial": 1,
   "hotkeys": {
    "equation": "Ctrl-E",
    "itemize": "Ctrl-I"
   },
   "labels_anchors": false,
   "latex_user_defs": false,
   "report_style_numbering": false,
   "user_envs_cfg": false
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {},
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}