CBSE CS and IP

CBSE Class 11 & 12 Computer Science and Informatics Practices Python Materials, Video Lecture

Tokens In Python


Tokens In Python



Tokens in Python -:

Today we will discuss "Python tokens", "Types of tokens with example", "List of tokens in Python", "How many types of tokens are allowed in Python".The Smallest individual unit in a python program is known as "Tokens".Python has mainly 5 types of token which are given below -:
  1. Keywords
  2. Identifiers
  3. Operators
  4. Literals 
  5. Punctuators 

Keywords in Python

What are the keywords in Python??

Keywords are the reserved word having special meaning and purpose in Python. Keywords cannot be used as an identifier name such as any variable name or Function Name. They are used to defined "Syntax" or "Structure" of the Python language. Python Keywords are case sensitive.

Python keywords is a special word, that forms the vocabulary of the python language. It is a reserved word that can not be used as an identifier.

How many keywords are there in Python??

There are as many as 33 keywords are used in Python programming language version 3.7. Keyword builds the vocabulary of the python language, they represent the "syntax and structure" of a python program.

How do you find a keyword in Python??

Following is the list of keywords in the python, there are 33 Keywords:

S No Keywords Description
1 and A logical operator
2 as To create an alias
3 assert For debugging
4 break To break out of a loop
5 class To define a class
6 continue It skips the remaining part of the loop and goes to the next iteration
7 def To define a function
8 del To delete an object
9 elif Used in conditional statements, same as else if
10 else Used in conditional statements
11 except It is used with exceptions. this part will execute if an exception occurs.
12 FALSE Boolean value, the result of comparison operations
13 finally It is used with exceptions. It executes irrespective of exception occurs or not
14 for To create a for loop
15 from To import specific parts of a module
16 global To declare a global variable
17 if To make a conditional statement
18 import To import a module
19 in To check if a value is present in a list, tuple, etc.
20 is To test if two variables are equal
21 lambda To create an anonymous function
22 None Represents a null value
23 nonlocal To declare a non-local variable
24 not A logical operator
25 or A logical operator
26 pass It is a "do nothing" statement. 
27 raise To raise an exception
28 return To exit a function and return a value
29 TRUE Boolean value, result of comparison operations
30 try To make a try...except statement
31 while To create a while loop
32 with Used to simplify exception handling
33 yield To end a function returns a generator
  

Identifier in Python

Before knowing Python Identifiers, we will learn what is Identity?? 

Identity is a property that makes a person different, it may be personality looks or expressions. Same way, Python identifier is the same as the identity of a Person.

Names given to a Class, Function or Variable is called Identifier in python. It helps to differentiate one entity from others.

What are the rules for identifiers??

  1. Identifier first letter should be any letter or underscore(_).
  2. Upper and lower-case letters are different.
  3. The digit 0 through 9  can be part of the identifier except for the first character.
  4. It should not be a keyword (list of 32 keywords discussed above).
  5. An identifier can not contain any special character except for underscore (_).

Valid / Invalid  Identifiers Examples -:

Valid Identifier -:  Mybook, file 123, z2td, _no
Invalid Identifier  -: 2rno , break , .mybook

Let us discuss why the given identifiers are Invalid.

  • 2rno: It is starting with a number
  • break: It is a Keyword
  • .mybook: It is containing "." keyword.  

Operators in Python

Operators are tokens that trigger some computation/action when applies to variables and other objects in an expression. Variables and objects to which the computation is applied are called operands.

There are following types of Operators:
  1. Arithmetic Operators
  2. Relational Operators
  3. Assignment Operators
  4. Logical Operators
  5. Bitwise operators
  6. Membership Operators
  7. Identity Operators

Arithmetic Operators -:

  • Arithmetic Operators are used to performing arithmetic operations like Addition, Subtraction, Multiplication, division etc.
  • There are following types of Arithmetic Operators in Python:
    Operators Description Example
           + perform addition of two number a+b
            - perform subtraction of two number a-b
            / perform division of two number a/b
            * perform multiplication of two number a*b
            % Modulus = returns remainder a%b


Relational (Comparison )Operators -:

  • Relationa operators perform an operation on data and return the result in Boolean true or False.
  • Relational Operators are used to comparing the value and decide the relation among them.
 Operators Description Example
      == True if a is equal to b a == b
      != True is a is not equal to b, otherwise False a != b
       > True if a is greater than b, otherwise False a>b
      >= Greater than or equal to, return true if a is greater than b or a is equals to b a>=b
       < True if a is less than b, otherwise False a < b
     <= Less than or equal to, return true if a is less than b or a is equals to b a<=

Assignment Operators -:

  • Assignment Operators are used to assigning value to a variable.
  • There are following types of assignment operators are there in Python:
Operators Description Example
      = Right side value will be assigned to the left side. a=b
    += both the operands will be added and result will be assigned to the left side operand. a+=b
     /= left operand divided by right operand, the result will be assigned to the left side operand a/=b
     *= both the operands will be multiplied and the result will be assigned to the left side operand. A*=b
     -= left operand minus right operand, the result will be assigned to the left side operand A-=b
    %= left operand modulus right operand, the result will be assigned to the left side operand a%=b
    //= left operand floor divide right operand, the result will be assigned to the left side operand a//=b
   **= the left operand to the power of right operand, the result will be assigned to the left side operand. a**=b

Logical Operators -:

  • Logical operators are used with conditional statements that can be TRUE or False.
  • Logical Operators are used to performing logical operations on the two given variables or values.
  • and, or and not are logical operators.
Operators Description Example
    and return true if both conditions are true x and y
     or return true if either or both condition is true x or y
     not reverse the condition not(a>b)

Bitwise Operators -:

  • Bitwise Operators acts on operands as if they were string or binary digits. They operate bit by bit hence its name is bitwise Operator.
Operator Meaning Example
          & Bitwise AND x & y = 0 (0000 0000)
           | Bitwise OR x | y = 14 (0000 1110)
           ~ Bitwise NOT ~x = -11 (1111 0101)
           ^ Bitwise XOR x ^ y = 14 (0000 1110)
         >> Bitwise right shift x >> 2 = 2 (0000 0010)
        << Bitwise left shift x << 2 = 40 (0010 1000)

Membership Operators -:

  • The Membership operators are used to validate whether a value is found within a sequence such as string, lists or tuples.

  • Operators Description Example
          in return true if a value exists in the sequence, else false. a in list
       not in return true if the value does not exist in the sequence, else false. a not in list

Identity Operators -:

  • Identity operators are used to comparing the memory location of two objects. 
    Operators Description Example
    is returns true if two variables point the same object, else false a is b
    is not returns true if two variables point the different object, else false a is not b


Literal (Value) in Python

  • Literals like a constant (often referred to as constant - values) are data items that have a fixed value.
  • Literals yields an object of the given type (String, integer, Long integer, Long floating number, Complex number ) with the given value.
    Python allows several kinds of literals.
    1. String Literals
    2. Numeric Literals
    3. Boolean Literals.

SN Literal Name Description
1 String Literal A string literal is a sequence of characters surrounded by quotes (single or double or triple quotes)
Example-: 'a' , 'abc' , "abc"
2 Numeric Literal 1. int (signed integers) -: No decimal points.
Example -: age=22
2. float (floating point real values) –: represent real number & written a decimal point.
Example -: height = 5.2
3. complex (complex numbers)
Example-: name = None
3 Boolean Literal 1. true (Boolean Literal)
2. false (Boolean Literal)

Punctuators  in Python

  • Punctuators is used to implement the grammar and structure of syntax. 
  • Punctuators are symbols that are used in a programming language to organize programming sentence.
The most common punctuators of Python programming language are:-  

| ' " # \ [ ] : ; =
( ) { } @ <<= >>= **= %= &=

2 comments: