
import pandas as pd
import numpy as np


# DataFrame 생성시 index, column 설정안하면 정수형 숫자로 기본값이 입력된다.
df = pd.DataFrame(np.random.randn(10,3)) # 랜덤값 10x3개의 np array객체입력
df.columns = ["one", "two", "three"]
df


# columns 리스트가 매우 많을때 쉽게 나열해 볼수 있는 방법
df_col_check = pd.DataFrame(df.columns)
df_col_check


# shift + tab + tab 클릭시 api정보 확인 할 수 있으나
# 가끔 안뜨는 경우가 있어서 필요시 print해서 확인 한다.

print(df.loc.__doc__)

Access a group of rows and columns by label(s) or a boolean array.

``.loc[]`` is primarily label based, but may also be used with a
boolean array.

Allowed inputs are:

- A single label, e.g. ``5`` or ``'a'``, (note that ``5`` is
  interpreted as a *label* of the index, and **never** as an
  integer position along the index).
- A list or array of labels, e.g. ``['a', 'b', 'c']``.
- A slice object with labels, e.g. ``'a':'f'``.

  .. warning:: Note that contrary to usual python slices, **both** the
      start and the stop are included

- A boolean array of the same length as the axis being sliced,
  e.g. ``[True, False, True]``.
- An alignable boolean Series. The index of the key will be aligned before
  masking.
- An alignable Index. The Index of the returned selection will be the input.
- A ``callable`` function with one argument (the calling Series or
  DataFrame) and that returns valid output for indexing (one of the above)

See more at :ref:`Selection by Label <indexing.label>`.

Raises
------
KeyError
    If any items are not found.
IndexingError
    If an indexed key is passed and its index is unalignable to the frame index.

See Also
--------
DataFrame.at : Access a single value for a row/column label pair.
DataFrame.iloc : Access group of rows and columns by integer position(s).
DataFrame.xs : Returns a cross-section (row(s) or column(s)) from the
    Series/DataFrame.
Series.loc : Access group of values using labels.

Examples
--------
**Getting values**

>>> df = pd.DataFrame([[1, 2], [4, 5], [7, 8]],
...      index=['cobra', 'viper', 'sidewinder'],
...      columns=['max_speed', 'shield'])
>>> df
            max_speed  shield
cobra               1       2
viper               4       5
sidewinder          7       8

Single label. Note this returns the row as a Series.

>>> df.loc['viper']
max_speed    4
shield       5
Name: viper, dtype: int64

List of labels. Note using ``[[]]`` returns a DataFrame.

>>> df.loc[['viper', 'sidewinder']]
            max_speed  shield
viper               4       5
sidewinder          7       8

Single label for row and column

>>> df.loc['cobra', 'shield']
2

Slice with labels for row and single label for column. As mentioned
above, note that both the start and stop of the slice are included.

>>> df.loc['cobra':'viper', 'max_speed']
cobra    1
viper    4
Name: max_speed, dtype: int64

Boolean list with the same length as the row axis

>>> df.loc[[False, False, True]]
            max_speed  shield
sidewinder          7       8

Alignable boolean Series:

>>> df.loc[pd.Series([False, True, False],
...        index=['viper', 'sidewinder', 'cobra'])]
            max_speed  shield
sidewinder          7       8

Index (same behavior as ``df.reindex``)

>>> df.loc[pd.Index(["cobra", "viper"], name="foo")]
       max_speed  shield
foo
cobra          1       2
viper          4       5

Conditional that returns a boolean Series

>>> df.loc[df['shield'] > 6]
            max_speed  shield
sidewinder          7       8

Conditional that returns a boolean Series with column labels specified

>>> df.loc[df['shield'] > 6, ['max_speed']]
            max_speed
sidewinder          7

Callable that returns a boolean Series

>>> df.loc[lambda df: df['shield'] == 8]
            max_speed  shield
sidewinder          7       8

**Setting values**

Set value for all items matching the list of labels

>>> df.loc[['viper', 'sidewinder'], ['shield']] = 50
>>> df
            max_speed  shield
cobra               1       2
viper               4      50
sidewinder          7      50

Set value for an entire row

>>> df.loc['cobra'] = 10
>>> df
            max_speed  shield
cobra              10      10
viper               4      50
sidewinder          7      50

Set value for an entire column

>>> df.loc[:, 'max_speed'] = 30
>>> df
            max_speed  shield
cobra              30      10
viper              30      50
sidewinder         30      50

Set value for rows matching callable condition

>>> df.loc[df['shield'] > 35] = 0
>>> df
            max_speed  shield
cobra              30      10
viper               0       0
sidewinder          0       0

**Getting values on a DataFrame with an index that has integer labels**

Another example using integers for the index

>>> df = pd.DataFrame([[1, 2], [4, 5], [7, 8]],
...      index=[7, 8, 9], columns=['max_speed', 'shield'])
>>> df
   max_speed  shield
7          1       2
8          4       5
9          7       8

Slice with integer labels for rows. As mentioned above, note that both
the start and stop of the slice are included.

>>> df.loc[7:9]
   max_speed  shield
7          1       2
8          4       5
9          7       8

**Getting values with a MultiIndex**

A number of examples using a DataFrame with a MultiIndex

>>> tuples = [
...    ('cobra', 'mark i'), ('cobra', 'mark ii'),
...    ('sidewinder', 'mark i'), ('sidewinder', 'mark ii'),
...    ('viper', 'mark ii'), ('viper', 'mark iii')
... ]
>>> index = pd.MultiIndex.from_tuples(tuples)
>>> values = [[12, 2], [0, 4], [10, 20],
...         [1, 4], [7, 1], [16, 36]]
>>> df = pd.DataFrame(values, columns=['max_speed', 'shield'], index=index)
>>> df
                     max_speed  shield
cobra      mark i           12       2
           mark ii           0       4
sidewinder mark i           10      20
           mark ii           1       4
viper      mark ii           7       1
           mark iii         16      36

Single label. Note this returns a DataFrame with a single index.

>>> df.loc['cobra']
         max_speed  shield
mark i          12       2
mark ii          0       4

Single index tuple. Note this returns a Series.

>>> df.loc[('cobra', 'mark ii')]
max_speed    0
shield       4
Name: (cobra, mark ii), dtype: int64

Single label for row and column. Similar to passing in a tuple, this
returns a Series.

>>> df.loc['cobra', 'mark i']
max_speed    12
shield        2
Name: (cobra, mark i), dtype: int64

Single tuple. Note using ``[[]]`` returns a DataFrame.

>>> df.loc[[('cobra', 'mark ii')]]
               max_speed  shield
cobra mark ii          0       4

Single tuple for the index with a single label for the column

>>> df.loc[('cobra', 'mark i'), 'shield']
2

Slice from index tuple to single label

>>> df.loc[('cobra', 'mark i'):'viper']
                     max_speed  shield
cobra      mark i           12       2
           mark ii           0       4
sidewinder mark i           10      20
           mark ii           1       4
viper      mark ii           7       1
           mark iii         16      36

Slice from index tuple to index tuple

>>> df.loc[('cobra', 'mark i'):('viper', 'mark ii')]
                    max_speed  shield
cobra      mark i          12       2
           mark ii          0       4
sidewinder mark i          10      20
           mark ii          1       4
viper      mark ii          7       1

Please see the :ref:`user guide<advanced.advanced_hierarchical>`
for more details and explanations of advanced indexing.


df.loc[1]

one     -0.896467
two      0.323917
three   -0.015165
Name: 1, dtype: float64


#df.loc[ df.index == 9 , :] # row값 9인 모든 column값, : => min:max 으로 All을 의미함
df.loc[ 0:1, :]


df.loc[0:3, ['one', 'two']]


print(df.iloc.__doc__)

Purely integer-location based indexing for selection by position.

``.iloc[]`` is primarily integer position based (from ``0`` to
``length-1`` of the axis), but may also be used with a boolean
array.

Allowed inputs are:

- An integer, e.g. ``5``.
- A list or array of integers, e.g. ``[4, 3, 0]``.
- A slice object with ints, e.g. ``1:7``.
- A boolean array.
- A ``callable`` function with one argument (the calling Series or
  DataFrame) and that returns valid output for indexing (one of the above).
  This is useful in method chains, when you don't have a reference to the
  calling object, but would like to base your selection on some value.
- A tuple of row and column indexes. The tuple elements consist of one of the
  above inputs, e.g. ``(0, 1)``.

``.iloc`` will raise ``IndexError`` if a requested indexer is
out-of-bounds, except *slice* indexers which allow out-of-bounds
indexing (this conforms with python/numpy *slice* semantics).

See more at :ref:`Selection by Position <indexing.integer>`.

See Also
--------
DataFrame.iat : Fast integer location scalar accessor.
DataFrame.loc : Purely label-location based indexer for selection by label.
Series.iloc : Purely integer-location based indexing for
               selection by position.

Examples
--------
>>> mydict = [{'a': 1, 'b': 2, 'c': 3, 'd': 4},
...           {'a': 100, 'b': 200, 'c': 300, 'd': 400},
...           {'a': 1000, 'b': 2000, 'c': 3000, 'd': 4000 }]
>>> df = pd.DataFrame(mydict)
>>> df
      a     b     c     d
0     1     2     3     4
1   100   200   300   400
2  1000  2000  3000  4000

**Indexing just the rows**

With a scalar integer.

>>> type(df.iloc[0])
<class 'pandas.core.series.Series'>
>>> df.iloc[0]
a    1
b    2
c    3
d    4
Name: 0, dtype: int64

With a list of integers.

>>> df.iloc[[0]]
   a  b  c  d
0  1  2  3  4
>>> type(df.iloc[[0]])
<class 'pandas.core.frame.DataFrame'>

>>> df.iloc[[0, 1]]
     a    b    c    d
0    1    2    3    4
1  100  200  300  400

With a `slice` object.

>>> df.iloc[:3]
      a     b     c     d
0     1     2     3     4
1   100   200   300   400
2  1000  2000  3000  4000

With a boolean mask the same length as the index.

>>> df.iloc[[True, False, True]]
      a     b     c     d
0     1     2     3     4
2  1000  2000  3000  4000

With a callable, useful in method chains. The `x` passed
to the ``lambda`` is the DataFrame being sliced. This selects
the rows whose index label even.

>>> df.iloc[lambda x: x.index % 2 == 0]
      a     b     c     d
0     1     2     3     4
2  1000  2000  3000  4000

**Indexing both axes**

You can mix the indexer types for the index and columns. Use ``:`` to
select the entire axis.

With scalar integers.

>>> df.iloc[0, 1]
2

With lists of integers.

>>> df.iloc[[0, 2], [1, 3]]
      b     d
0     2     4
2  2000  4000

With `slice` objects.

>>> df.iloc[1:3, 0:3]
      a     b     c
1   100   200   300
2  1000  2000  3000

With a boolean array whose length matches the columns.

>>> df.iloc[:, [True, False, True, False]]
      a     c
0     1     3
1   100   300
2  1000  3000

With a callable function that expects the Series or DataFrame.

>>> df.iloc[:, lambda df: [0, 2]]
      a     c
0     1     3
1   100   300
2  1000  3000


df.iloc[2]

one      1.093343
two     -2.020293
three    0.099026
Name: 2, dtype: float64


df.iloc[1, 2] # row 1, cal 2 에 해당하는 값

-0.015164940108393593


#df.iloc[:, 0:3]
df.iloc[:3, :2] # 0~3, 0~2, df.iloc[0:3, 0:2] 와 같음


# Column 조건문으로 행 추출하기
df["two"].values > 0.3
#df[df["two"].values > 0.3]
#df["two"].isin([-0.643932, 0.417865])

array([False,  True, False, False, False,  True,  True, False, False,
       False])


df["four"] = [np.nan, np.nan, np.nan, 4, 5, 6, 7, 8, 9, 10]
df


df.dropna() # 데이터 내에 있는 모든 결측치 제거


df.fillna(value=-1) # nan값에 값 넣기


df.isnull()


df.isnull()["four"]

0     True
1     True
2     True
3    False
4    False
5    False
6    False
7    False
8    False
9    False
Name: four, dtype: bool


df.loc[df.isnull()["four"],:]


df["four"].unique() # 특정 column에 중복값을 제외한 unique한 값 array

array([nan,  4.,  5.,  6.,  7.,  8.,  9., 10.])


np.unique(df["four"])

array([ 4.,  5.,  6.,  7.,  8.,  9., 10., nan])


len(np.unique(df["four"]))

8


mydict = [{'a': 1, 'b': 2, 'c': 3, 'd': 4},
          {'a': 100, 'b': 200, 'c': 300, 'd': 400},
          {'a': 1000, 'b': 2000, 'c': 3000, 'd': 4000 }]
df_dict = pd.DataFrame(mydict)
df_dict


df_dict = df_dict.drop(0, axis=0)
df_dict


df_dict = df_dict.drop("b", axis=1)
df_dict


import pandas as pd
import numpy as np

# pandas 에는 Series, DataFrame, ...


sr = pd.Series([1,2,3,4,5])
sr

0    1
1    2
2    3
3    4
4    5
dtype: int64


# Series의 값
sr.values

array([1, 2, 3, 4, 5], dtype=int64)


# Series의 index
sr.index

RangeIndex(start=0, stop=5, step=1)


# Series의 DataType
sr.dtypes

dtype('int64')


sr2 = pd.Series([1,2,3,4,5],index=['A','B','C','D','E'])
sr2

A    1
B    2
C    3
D    4
E    5
dtype: int64


sr.index=['A','B','C','D','E'] #와 같은 효과
sr

A    1
B    2
C    3
D    4
E    5
dtype: int64


sr["A"]=100 # "A" index에 value 100 값을 넣기
sr

A    100
B      2
C      3
D      4
E      5
dtype: int64


ser = pd.Series([1,2,3,4,5,6,7,8,9,10])


ser_sum = ser.sum() #합계
ser_min = ser.min() #최소값
ser_max = ser.max() #최대값
ser_sum, ser_min, ser_max

(55, 1, 10)


ser_mean = ser.mean() #평균
ser_std = ser.std() #표준편차
ser_skew = ser.skew() #왜도
ser_kurt = ser.kurt() #첨도

ser_mean, ser_std, ser_skew, ser_kurt

(5.5, 3.0276503540974917, 0.0, -1.2000000000000002)


ser_unique = ser.unique() #고유값
ser_idxmax = ser.idxmax() #최대값의 위치
ser_idxmin = ser.idxmin() #최소값의 위치
ser_isin = ser.isin([1,2,3,4]) #포함 여부 확인

ser_unique, ser_idxmax, ser_idxmin, ser_isin

(array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10], dtype=int64),
 9,
 0,
 0     True
 1     True
 2     True
 3     True
 4    False
 5    False
 6    False
 7    False
 8    False
 9    False
 dtype: bool)


ser.isin([6,7]).sum() # True =1 , False =0 으로 연산이 가능하여 sum() 을 활용 가능

2


import pandas as pd
import numpy as np
# pandas 에는 Series, DataFrame, ...


data = {"names": ["kim", "lee", "park", "choi", "moon", "ahn"],
        "year": [2023, 2022, 2021, 2020, 2019, 2018],
        "score": [87, 73, 70, 92, 56, 50] }

df = pd.DataFrame(data, columns=["year", "names", "score", "note"],
                          index=[1,2,3,4,5,6])
df = pd.DataFrame(data, columns=["year", "names", "score", "note"],
                          index=range(1,7)) #index=[1,2,3,4,5,6] 과 같음
df


df.head(2) # Table값이 너무 큰 경우 앞쪽 2개 목록만 확인
df.tail(2) # Table값이 너무 큰 경우 뒤쪽 2개 목록만 확인


df.values

array([[2023, 'kim', 87, nan],
       [2022, 'lee', 73, nan],
       [2021, 'park', 70, nan],
       [2020, 'choi', 92, nan],
       [2019, 'moon', 56, nan],
       [2018, 'ahn', 50, nan]], dtype=object)


df.index

RangeIndex(start=1, stop=7, step=1)


df.columns

Index(['year', 'names', 'score', 'note'], dtype='object')


df.dtypes

year      int64
names    object
score     int64
note     object
dtype: object

df


df.index.name = "NUM"
df.columns.name = "User"
df


df["year"] # df.year 으로 사용해도 동일

NUM
1    2023
2    2022
3    2021
4    2020
5    2019
6    2018
Name: year, dtype: int64


df2 = df[["names", "score"]].copy()
df2


#df["note"] = "O" #으로 입력하면 전체 값 "O" 으로 입력
df["note"] = ["O", "X", "O", "O", "X", "X"]
df


df["point"] = np.arange(6)
#df["address"] = range(6) 와 같음
df


# Series로 특정 행값을 해당하는 index에 추가
val = pd.Series([-1.2, -1.5, -1.7], index=[1, 2, 6])
val

1   -1.2
2   -1.5
6   -1.7
dtype: float64


df.point = val # Series 값에 없는 값들은 NaN 값으로 변경됨
df


val2 = np.arange(5,11) # np.arange 입력시 행(row)크기에 맞춰서 넣어줘야만 반영 됨
val2

array([ 5,  6,  7,  8,  9, 10])


df.point = val2
df


df["high_score"] = df["score"] > 80
df


df['test'] = True
df


#df = df.replace({"O":1,"X":0})
#df = df.replace({True:1,False:0})

#high_score에 해당하는 값만 replace
df['high_score'] = df['high_score'].replace({True:1,False:0})
df


del df['test']
del df['point']
df


df[:3] # df[0:3] 와 같음


#df.index = np.arange(0,6)
df.index = pd.Series(["a","b","c","d","e","f"])
df


# loc은 location 'b'에 해당하는 행
df.loc['b']

User
year          2022
names          lee
score           73
note             X
high_score       0
Name: b, dtype: object


df.loc['b':'e'] # df['b':'e'] 와 같으나 확장성을 위해 loc 를 사용하도록 한다


df.loc['b':'e', 'score']

b    73
c    70
d    92
e    56
Name: score, dtype: int64


df.loc[:,'names'] # == df['names']

a     kim
b     lee
c    park
d    choi
e    moon
f     ahn
Name: names, dtype: object


# iloc은 index 번호를 사용한 location
df.iloc[3] # 3번째 행

User
year          2020
names         choi
score           92
note             O
high_score       1
Name: d, dtype: object


df.iloc[0:3, 0:2]


df.iloc[:,1:2] #첫번째 열만 출력


df.index = np.arange(1,7)
df['score'] > 70

1     True
2     True
3    False
4     True
5    False
6    False
Name: score, dtype: bool


#score가 70점보다 큰 모든 행의 값
df.loc[df['score'] > 70, :]
#df.loc[df['score'] > 70, ['names','score','note']]


#df.loc[ 조건문 , columns 선택 ]
df.loc[df['names'] == 'choi', ['names', 'score']]


#df.loc[ 조건문 , columns 선택 ]
#df.loc[ (df["score"]>50) & (df["note"]=="X"), ["names","score"]]
df.loc[(df['score'] > 70) & (df['note']=="X"),:]


df.loc[df['score'] > 70, 'note'] = 'O'
df


# 새로운 행 삽입
df.loc['7',:] = [2017,'ryu',37, "O", False]
df['high_score'] = df['high_score'].replace({True:1,False:0})
df

	one	two	three
0	-1.141389	-1.597709	-0.593118
1	-0.896467	0.323917	-0.015165
2	1.093343	-2.020293	0.099026
3	1.971081	-0.379402	0.751828
4	-0.569713	-0.061500	-0.489205
5	-0.502000	0.545283	0.037430
6	1.407763	2.647069	0.217019
7	-1.627089	-0.195426	0.157814
8	-1.544826	-1.033046	-0.704031
9	-0.919720	-0.968175	-1.126654

	one	two	three	four
0	-1.141389	-1.597709	-0.593118	NaN
1	-0.896467	0.323917	-0.015165	NaN
2	1.093343	-2.020293	0.099026	NaN
3	1.971081	-0.379402	0.751828	4.0
4	-0.569713	-0.061500	-0.489205	5.0
5	-0.502000	0.545283	0.037430	6.0
6	1.407763	2.647069	0.217019	7.0
7	-1.627089	-0.195426	0.157814	8.0
8	-1.544826	-1.033046	-0.704031	9.0
9	-0.919720	-0.968175	-1.126654	10.0

	one	two	three	four
3	1.971081	-0.379402	0.751828	4.0
4	-0.569713	-0.061500	-0.489205	5.0
5	-0.502000	0.545283	0.037430	6.0
6	1.407763	2.647069	0.217019	7.0
7	-1.627089	-0.195426	0.157814	8.0
8	-1.544826	-1.033046	-0.704031	9.0
9	-0.919720	-0.968175	-1.126654	10.0

	one	two	three	four
0	-1.141389	-1.597709	-0.593118	-1.0
1	-0.896467	0.323917	-0.015165	-1.0
2	1.093343	-2.020293	0.099026	-1.0
3	1.971081	-0.379402	0.751828	4.0
4	-0.569713	-0.061500	-0.489205	5.0
5	-0.502000	0.545283	0.037430	6.0
6	1.407763	2.647069	0.217019	7.0
7	-1.627089	-0.195426	0.157814	8.0
8	-1.544826	-1.033046	-0.704031	9.0
9	-0.919720	-0.968175	-1.126654	10.0

User	year	names	score	note	high_score
1	2023.0	kim	87.0	O	1
2	2022.0	lee	73.0	O	0
3	2021.0	park	70.0	O	0
4	2020.0	choi	92.0	O	1
5	2019.0	moon	56.0	X	0
6	2018.0	ahn	50.0	X	0
7	2017.0	ryu	37.0	O	0

	one	two	three	four
0	False	False	False	True
1	False	False	False	True
2	False	False	False	True
3	False	False	False	False
4	False	False	False	False
5	False	False	False	False
6	False	False	False	False
7	False	False	False	False
8	False	False	False	False
9	False	False	False	False

	year	names	score	note
1	2023	kim	87	NaN
2	2022	lee	73	NaN
3	2021	park	70	NaN
4	2020	choi	92	NaN
5	2019	moon	56	NaN
6	2018	ahn	50	NaN

User	year	names	score	note	point
NUM
1	2023	kim	87	O	-1.2
2	2022	lee	73	X	-1.5
3	2021	park	70	O	NaN
4	2020	choi	92	O	NaN
5	2019	moon	56	X	NaN
6	2018	ahn	50	X	-1.7

IT GOGO

엔트로피란 (Entropy)

Entropy

Python Pandas / loc, iloc, Nan, unique, drop

Python Pandas DataFrame 데이터 활용2

loc vs iloc 비교, Nan (결측값), unique, drop

BASIC_03.Pandas_DataFrame.ipynb¶

loc vs iloc¶

─────────────────────────────────────¶

NaN 결측값 다루기¶

unique¶

drop¶

Python Pandas 기본 (Series, DataFrame)

파이썬 라이브러리 pandas

1. Pandas 란?

2. Series 기본

BASIC_01.Pandas¶

Seriese 사용해보기¶

Series의 생성 및 중요 Method¶

3. DataFrame 기본

BASIC_01.Pandas¶

DataFrame 사용해보기¶

DataFrame loc, iloc 사용¶

DataFrame 의 bool type 다루기¶

Python Jupyter 설치

파이썬 주피터 노트북 설치 (jupyter notebook)

1. 파이썬 3.9x 설치

2. 주피터 노트북 설치

3. 아나콘다 설치

4. Python 유용한 라이브러리 설치 (pandas, numpy, scikit-learn, statsmodels)

[Windows] 폴더 읽기 전용 설정 변경되지 않을때

[EP2] FreeFileSync 실시간 자동 동기화

[EP2] FreeFileSync 실시간 자동 동기화

[EP1] FreeFileSync 폴더 동기화

[EP1] FreeFileSync 폴더 동기화

[EP1] 나만의 Cloud서비스 만들기

[EP1] 나만의 Cloud서비스 만들기

[Windows] 네트워크 공유 폴더 "액세스 권한이 없습니다" 해결 방법

[Windows] 네트워크 공용 폴더 및 파일 공유하기

[Algorithms] 후위수식 변환 및 예제코드

[Stack] 전위수식(prefix), 후위수식(postfix), 중위수식(infix)

[Stack] 전위수식(prefix), 후위수식(postfix), 중위수식(infix)

[C/C++] register int 와 int 의 수행속도 차이 비교

register int 와 int 의 수행속도 차이 비교

[GoPro] 영상 파일명 시간순으로 변경