data-engineering-interview-questions

<h1 align="center">准备数据工程师面试的2000多个问题。</h1> <h2 align="center"><a href="./content/full.md">问题完整列表</a></h2> <h1 align="center">数据工程师面试问题</h1> <div> <table> <tr> <tr> <th colspan="5">数据库和数据仓库</th> </tr> <tr> <th>GitHub仓库</th> <th>官方页面</th> <th>问题</th> <th>描述</th> <th>有用链接</th> </tr> <tr> <th><a href="https://github.com/apache/cassandra"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="Cassandra"></a></th> <th><a href="https://cassandra.apache.org/_/index.html"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/cassandra.ico" alt="Cassandra"></a></th> <th><a href="./content/cassandra.md">Apache Cassandra</a></th> <th>Cassandra是一个分布式、宽列存储的NoSQL数据库管理系统。</th> <th><a href="https://github.com/Anant/awesome-cassandra">Awesome Cassandra</a></th> </tr> <tr> <th><a href="https://github.com/greenplum-db/gpdb"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="Greenplum"></a></th> <th><a href="https://greenplum.org/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/greenplum.ico" alt="Greenplum"></a></th> <th><a href="./content/greenplum.md">Greenplum</a></th> <th>Greenplum是一个基于MPP架构和Postgres开源数据库技术的大数据技术。</th> <th><a href="https://github.com/kongyew/awesome-greenplum">Awesome Greenplum</a></th> </tr> <tr> <th><a href="https://github.com/mongodb/mongo"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="MongoDB"></a></th> <th><a href="https://www.mongodb.com/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/mongo.ico" alt="MongoDB"></a></th> <th><a href="./content/mongo.md">MongoDB</a></th> <th>MongoDB是一个面向文档的数据库。</th> <th><a href="https://github.com/ramnes/awesome-mongodb">Awesome MongoDB</a></th> </tr> <tr> <th><a href="https://github.com/apache/hbase"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="Hbase"></a></th> <th><a href="https://hbase.apache.org/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/hbase.ico" alt="Hbase"></a></th> <th><a href="./content/hbase.md">Apache Hbase</a></th> <th>HBase是一个开源的非关系型分布式数据库。</th> <th><a href="https://github.com/rayokota/awesome-hbase">Awesome HBase</a></th> </tr> <tr> <th><a href="https://github.com/apache/hive"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="Hive"></a></th> <th><a href="https://hive.apache.org/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/hive.ico" alt="Hive"></a></th> <th><a href="./content/hive.md">Apache Hive</a></th> <th>Apache Hive是一个建立在Apache Hadoop之上的数据仓库软件项目，用于提供数据查询和分析。</th> <th><a href="https://github.com/dharmeshkakadia/awesome-hive">Awesome Hive</a></th> </tr> <tr> <th colspan="2"><a href="https://aws.amazon.com/dynamodb/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/dynamodb.ico" alt="Amazon DynamoDB"></a></th> <th><a href="./content/dynamodb.md">Amazon DynamoDB</a></th> <th>Amazon DynamoDB是一个完全托管的专有NoSQL数据库服务。</th> <th><a href="https://github.com/alexdebrie/awesome-dynamodb">Awesome DynamoDB</a> <a href="https://github.com/donnemartin/awesome-aws">Awesome AWS</a></th> </tr> <tr> <th colspan="2"><a href="https://aws.amazon.com/redshift"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/redshift.ico" alt="Amazon Redshift"></a></th> <th><a href="./content/redshift.md">Amazon Redshift</a></th> <th>Amazon Redshift是一个数据仓库产品。</th> <th><a href="https://github.com/awslabs/amazon-redshift-utils">Amazon Redshift Utilities</a> <a href="https://github.com/donnemartin/awesome-aws">Awesome AWS</a></th> </tr> <tr> <th colspan="2"><a href="https://cloud.google.com/bigquery"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/bigquery.ico" alt="BigQuery"></a></th> <th><a href="./content/bigquery.md">BigQuery GCP</a></th> <th>BigQuery是一个全托管、无服务器的数据仓库。</th> <th><a href="https://github.com/coty/awesome-bigquery">Awesome BigQuery</a></th> </tr> <tr> <th colspan="2"><a href="https://cloud.google.com/bigtable"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/bigtable.ico" alt="Bigtable"></a></th> <th><a href="./content/bigtable.md">Bigtable GCP</a></th> <th>Bigtable是一个完全托管的宽列和键值NoSQL数据库服务。</th> <th><a href="https://github.com/zrosenbauer/awesome-bigtable">Awesome Bigtable</a></th> </tr> <th colspan="5"><a></a></th> <tr> <th colspan="5">数据格式</th> </tr> <tr> <th><a href="https://github.com/apache/avro"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="Avro"></a></th> <th><a href="https://avro.apache.org/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/avro.ico" alt="Avro"></a></th> <th><a href="./content/avro.md">Apache Avro</a></th> <th>Avro是一个面向行的远程过程调用和数据序列化框架。</th> <th><a href="https://github.com/m0nhawk/awesome-avro">Awesome Avro</a></th> </tr> <tr> <th><a href="https://github.com/apache/parquet"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="Parquet"></a></th> <th><a href="https://parquet.apache.org/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/parquet.ico" alt="Parquet"></a></th> <th><a href="./content/parquet.md">Apache Parquet</a></th> <th>Apache Parquet是一种为高效数据存储和检索而设计的面向列的数据文件格式。</th> <th><a href="TODO">TODO</a></th> </tr> <tr> <th><a href="https://github.com/delta-io"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="Delta"></a></th> <th><a href="https://delta.io/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/deltalake.ico" alt="Delta"></a></th> <th><a href="./content/delta.md">Delta</a></th> <th>Delta Lake是一个存储框架，可以使用计算引擎构建Lakehouse架构</th> <th><a href="https://github.com/MrPowers/delta-examples">Delta examples</a></th> </tr> <th colspan="5"><a></a></th> <tr> <th colspan="5">大数据框架</th> </tr> <tr> <th><a href="https://github.com/apache/airflow"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="Airflow"></a></th> <th><a href="https://airflow.apache.org/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/airflow.ico" alt="Airflow"></a></th> <th><a href="./content/airflow.md">Apache Airflow</a></th> <th>Apache Airflow是一个用于数据工程管道的工作流管理平台。</th> <th><a href="https://github.com/jghoman/awesome-apache-airflow">Awesome Airflow</a></th> </tr> <tr> <th><a href="https://github.com/apache/flume"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="Flume"></a></th> <th><a href="https://flume.apache.org/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/flume.ico" alt="Flume"></a></th> <th><a href="./content/flume.md">Apache Flume</a></th> <th>Apache Flume是一个分布式、可靠且可用的软件，用于高效收集、聚合和移动大量日志数据。</th> <th><a href="TODO">TODO</a></th> </tr> <tr> <th><a href="https://github.com/apache/hadoop"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="Hadoop"></a></th> <th><a href="https://hadoop.apache.org/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/hadoop.ico" alt="Hadoop"></a></th> <th><a href="./content/hadoop.md">Apache Hadoop</a></th> <th>Apache Hadoop是一个软件工具集合，可以利用多台计算机的网络来解决涉及海量数据和计算的问题。</th> <th><a href="https://github.com/youngwookim/awesome-hadoop">Awesome Hadoop</a></th> </tr> <tr> <th><a href="https://github.com/apache/impala"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="Impala"></a></th> <th><a href="https://impala.apache.org/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/impala.ico" alt="Impala"></a></th> <th><a href="./content/impala.md">Apache Impala</a></th> <th>Apache Impala是一个用于查询存储在运行Apache Hadoop的计算机集群中数据的并行处理SQL查询引擎。</th> <th><a href="TODO">待办</a></th> </tr> <tr> <th><a href="https://github.com/apache/kafka"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="Kafka"></a></th> <th><a href="https://kafka.apache.org/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/kafka.ico" alt="Kafka"></a></th> <th><a href="./content/kafka.md">Apache Kafka</a></th> <th>Apache Kafka是一个分布式事件存储和流处理平台。</th> <th><a href="https://github.com/semantalytics/awesome-kafka">优秀的Kafka资源</a></th> </tr> <tr> <th><a href="https://github.com/apache/nifi"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="NiFi"></a></th> <th><a href="https://nifi.apache.org/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/nifi.ico" alt="NiFi"></a></th> <th><a href="./content/nifi.md">Apache NiFi</a></th> <th>Apache NiFi是一个旨在自动化软件系统之间数据流的软件项目。</th> <th><a href="https://github.com/jfrazee/awesome-nifi">优秀的NiFi资源</a></th> </tr> <tr> <th><a href="https://github.com/apache/spark"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="Spark"></a></th> <th><a href="https://spark.apache.org/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/spark.ico" alt="Spark"></a></th> <th><a href="./content/spark.md">Apache Spark</a></th> <th>Apache Spark是用于大规模数据处理的统一分析引擎。</th> <th><a href="https://github.com/awesome-spark/awesome-spark">优秀的Spark资源</a></th> </tr> <tr> <th><a href="https://github.com/apache/flink"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="Flink"></a></th> <th><a href="https://flink.apache.org/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/flink.ico" alt="Flink"></a></th> <th><a href="./content/flink.md">Apache Flink</a></th> <th>Apache Flink是统一的流处理和批处理框架。</th> <th><a href="https://github.com/wuchong/awesome-flink">优秀的Flink资源</a></th> </tr> <tr> <th><a href="https://github.com/kubernetes/kubernetes"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="Kubernetes"></a></th> <th><a href="https://kubernetes.io/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/kuber.ico" alt="Kubernetes"></a></th> <th><a href="./content/kubernetes.md">Kubernetes</a></th> <th>Kubernetes是一个用于在多个主机上管理容器化应用程序的系统。</th> <th><a href="https://github.com/ramitsurana/awesome-kubernetes">优秀的Kubernetes资源</a></th> </tr> <th colspan="5"><a></a></th> <tr> <th colspan="5">云服务提供商</th> </tr> <tr> <th><a href="https://github.com/aws"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="AWS"></a></th> <th><a href="https://aws.amazon.com/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/aws.ico" alt="AWS"></a></th> <th><a href="./content/aws.md">亚马逊网络服务</a></th> <th>亚马逊网络服务是一个提供可扩展和经济高效的云计算解决方案的在线平台。</th> <th><a href="https://github.com/donnemartin/awesome-aws">优秀的AWS资源</a></th> </tr> <tr> <th><a href="https://github.com/Azure"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="Azure"></a></th> <th><a href="https://azure.microsoft.com/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/azure.ico" alt="Azure"></a></th> <th><a href="./content/azure.md">微软Azure</a></th> <th>微软Azure是微软的公共云计算平台。</th> <th><a href="https://github.com/kristofferandreasen/awesome-azure">优秀的Azure资源</a></th> </tr> <tr> <th><a href="https://github.com/GoogleCloudPlatform"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="GCP"></a></th> <th><a href="https://cloud.google.com/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/gcp.ico" alt="GCP"></a></th> <th><a href="./content/gcp.md">谷歌云平台</a></th> <th>谷歌云平台是一套云计算服务。</th> <th><a href="https://github.com/GoogleCloudPlatform/awesome-google-cloud">优秀的GCP资源</a></th> </tr> <th colspan="5"><a></a></th> <tr> <th colspan="5"><b>理论</b></th> </tr> <tr> <th colspan="2"><a href="./content/dwha.md"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/dwha.ico" alt="DWHA"></a></th> <th><a href="./content/dwha.md">数据仓库架构</a></th> <th>数据仓库架构是一种定义企业内部最终客户计算的数据通信处理和展示的整体架构的方法。</th> <th><a href="https://github.com/numetriclabz/awesome-db">优秀的数据库资源</a></th> </tr> <tr> <th colspan="2"><a href="./content/data-structure.md"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/datastruct.ico" alt="Airflow"></a></th> <th><a href="./content/data-structure.md">数据结构</a></th> <th>数据结构是一种用于组织、处理、检索和存储数据的专门格式。</th> <th><a href="TODO">待办</a></th> </tr> <tr> <th colspan="2"><a href="./content/sql.md"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/sql.ico" alt="SQL"></a></th> <th><a href="./content/sql.md">SQL</a></th> <th>SQL是一种在编程中使用的领域特定语言，设计用于管理关系数据库管理系统（RDBMS）中保存的数据。</th> <th><a href="https://github.com/danhuss/awesome-sql">优秀的SQL资源</a></th> </tr> <th colspan="5"><a></a></th> <tr> <th colspan="5">数据可视化工具/商业智能</th> </tr> <tr> <th colspan="2"><a href="./content/tableau.md"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/tableau.ico" alt="Tableau"></a></th> <th><a href="./content/tableau.md">Tableau</a></th> <th>Tableau是商业智能中使用的强大数据可视化工具。</th> <th><a href="TODO">待办</a></th> </tr> <th colspan="2"><a href="./content/looker.md"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/looker.ico" alt="Looker"></a></th> <th><a href="./content/looker.md">Looker</a></th> <th>Looker是一个用于商业智能、数据应用和嵌入式分析的企业平台，帮助您实时探索和共享见解。</th> <th><a href="TODO">待办</a></th> </tr> <tr> <th><a href="https://github.com/apache/superset"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/github.ico" alt="Kafka"></a></th> <th colspan="2"><a href="https://superset.apache.org/"><img style="vertical-align:middle" src="https://raw.githubusercontent.com/OBenner/data-engineering-interview-questions/master/img/icon/superset.ico" alt="Apache Superset"></a></th> <th><a href="./content/superset.md">Apache Superset</a></th> <th>Superset是一个现代化的数据探索和数据可视化平台</th> <th><a href="TODO">待办</a></th> </tr> </table> </div> <div> <h2 align="center">贡献</h2> <h3>请为这个仓库做出贡献，以使其变得更好。任何变更，如新问题、代码改进、文档改进等都非常欢迎。</h3> </div>