Python
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
 
User Name:
Password:
Remember me
Go Back   Web Development Archives Mailing Lists Python

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Display Modes
 
Unread Web Development Archives Sponsor:
  #1  
Old May 16th, 2008, 12:52 PM
alivip alivip is offline
Registered User
Dev Archives Newbie (0 - 499 posts)
 
Join Date: Mar 2008
Posts: 4 alivip User rank is Just a Lowly Private (1 - 20 Reputation Level)  
Time spent in forums: 26 m 37 sec
Reputation Power: 0
How to modify my code to get every word & previos word from file?please help

I write code to get most frequent words in the file
I won't to implement bigram probability by modifying the code to do the following:
How can I get every Token (word) and PreviousToken(Previous word) and frequency and probability
From text file and put each one in cell in table

For example if the text file content is
"Every man has a price. Every woman has a price."

First Token(word) is "Every" PreviousToken(Previous word) is none(no previos)
Second Token(word) is "man" PreviousToken(Previous word) is "Every"
Third Token(word) is "has" PreviousToken(Previous word) is "man"
Forth Token(word) is "a" PreviousToken(Previous word) is "has"
Fifth Token(word) is "price" PreviousToken(Previous word) is "a"

Sixth Token(word) is "Every" PreviousToken(Previous word) is none(no previos)
Seventh Token(word) is "man" PreviousToken(Previous word) is "Every"
Eighth Token(word) is "has" PreviousToken(Previous word) is "man"
Ninth Token(word) is "a" PreviousToken(Previous word) is "has"
Tenth Token(word) is "price" PreviousToken(Previous word) is "a"


Frequency of "has a" is 2 (repeated two times first and second sentence)
Frequency of " a price" is 2 (repeated two times first and second sentence)
Frequency of "Every man" is 1 (occur one time only)
Frequency of "man has" is 1 (occur one time only)
Frequency of "Every woman" is 1 (occur one time only)
Frequency of "woman has" is 1 (occur one time only)

Probability of "has a" is 2/10 (Frequency of "has a" divided by all word )
Probability of "a price" is 2/10 (Frequency of "a price" divided by all word )
Probability of "Every man" is 1/10 (Frequency "Every man" divided by all word )

Probability of "man has" is 1/10 (Frequency of man has" divided by all word )

Probabilityof "Every woman" is 1/10 (Frequency of "Every woman" divided by all word )
Probability of "woman has" is 1/10 (Frequency of "woman has" divided by all word )


Code:
# a look at the Tkinter Text widget

# use ctrl+c to copy, ctrl+x to cut selected text,

# ctrl+v to paste, and ctrl+/ to select all
  # count words in a text and show the first ten items
 # by decreasing frequency

import Tkinter as tk
import os, glob
import sys
import string
import re
import tkFileDialog      
def most_frequant_word():    
 browser= tkFileDialog.askdirectory()
 #browser= os.listdir(a)

 word_freq = {}
 for root, dirs, files in os.walk(browser):
    #print 'Looking into %s' % root.split('\\')[-1]
    #print 'Found %d dirs and %d files' % (len(dirs), len(files))
    text1.insert(tk.INSERT, 'Found %d dirs and %d files' % (len(dirs), len(files)))
    text1.insert(tk.INSERT, "\n")
    for idx, file in enumerate(files):
     
     print 'File #%d: %s' % (idx + 1, file)
       #text1.insert(tk.INSERT, 'File #%d: %s' % (idx + 1, file))
       #text1.insert(tk.INSERT, "\n")
     ff = open (os.path.join(root, file), "r")
     text = ff.read ( )
     ff.close ( )
     #word_freq = {}     
     word_list = text.split()
     for word in word_list:
      word = word.lower()
      word = word.rstrip('.,/"\ -_;\[](){} ')

      #if word.isalpha():
                # build the dictionary
      count = word_freq.get(word, 0)
      word_freq[word] = count + 1
 
       # create a list of (freq, word) tuples
      freq_list = [(word,freq ) for freq,word  in word_freq.items()]
     
       # sort the list by the first element in each tuple (default)
      freq_list.sort(reverse=True)
     
     for n, tup in enumerate(freq_list):
    # print the first ten items
      if n < 5:
       if idx == 3:  
        print "%s times: %s" % tup
        text1.insert(tk.INSERT, "%s times: %s" % tup)
       #text1.insert(tk.INSERT, word)
        text1.insert(tk.INSERT, "\n")
        
# raw_input('\nHit enter to exit')
 
root = tk.Tk(className = " most_frequant_word")
# text entry field, width=width chars, height=lines text
v1 = tk.StringVar()
text1 = tk.Text(root, width=50, height=50, bg='green')
text1.pack()
# function listed in command will be executed on button click
button1 = tk.Button(root, text='Brows', command=most_frequant_word)
button1.pack(pady=5)
text1.focus()
root.mainloop()

Reply With Quote
Reply

Viewing: Web Development Archives Mailing Lists Python > How to modify my code to get every word & previos word from file?please help


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are Off
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 5 hosted by Hostway
Stay green...Green IT